{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个实例中，我们使用tensorflow来实现一个简单的手写数字识别的网络，并用这个网络来做个\n",
    "简单的识别示例。\n",
    "\n",
    "首先导入一些用到的库。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:50.515650Z",
     "start_time": "2018-06-01T06:32:43.133728Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:39:48.304594Z",
     "start_time": "2018-06-01T04:55:17.674707Z"
    }
   },
   "source": [
    "先来看看数据长什么样子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.075054Z",
     "start_time": "2018-06-01T06:32:50.518290Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-5-c9034f919bc1>:1: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./mnist_data/train-images-idx3-ubyte.gz\n",
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./mnist_data/train-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.one_hot on tensors.\n",
      "Extracting ./mnist_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting ./mnist_data/t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "(55000, 784)\n",
      "(55000, 10)\n",
      "(5000, 784)\n",
      "(5000, 10)\n",
      "(10000, 784)\n",
      "(10000, 10)\n"
     ]
    }
   ],
   "source": [
    "mnist = input_data.read_data_sets(\"./mnist_data/\", one_hot=True)\n",
    "\n",
    "print(mnist.train.images.shape)\n",
    "print(mnist.train.labels.shape)\n",
    "\n",
    "print(mnist.validation.images.shape)\n",
    "print(mnist.validation.labels.shape)\n",
    "\n",
    "print(mnist.test.images.shape)\n",
    "print(mnist.test.labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:49:40.128071Z",
     "start_time": "2018-06-01T05:49:40.123888Z"
    }
   },
   "source": [
    "可以看到images里面有数量不等的图片，每张图片是28x28长度的一个一维向量，\n",
    "所以用的时候需要先给它还原成28x28的二维图片。labels中则是图片对应的数字的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.695746Z",
     "start_time": "2018-06-01T06:32:51.077167Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAHiCAYAAADf3nSgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd8lEX+wPHvpBBKQleUXkNTyin2ggIWxN4b6qEo2DgV9fT8eZyn3lkBsVFU7PVQsWEFGwoiSpFepIP0HpLs/P5ImHlmzYbNZjeTbD7v18vXfWdndneOJ7vffWaemUdprQUAAJS9FN8dAACgsiIJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwhCQcoJTSSqkdSqn7o2zfTym1vfB5rRPdP5QMxzO5xHA8hxS210qptET3DyXHZ1REsVmHpZTSItJGa72wsHysiHwc1qyGiJyntX4n0vNQPhRxPOuLyHsi0k5EUkVkjojcprX+rrjnoXwo7rgopa4QkRdE5Bqt9ejA481FZImIpGut88qmp4hWUcdUKZUqIkNE5K8ikiUiC0XkBK315uKeV1Hx67AYWutvRCRzb1kp1V1ExovIJ776hFLZLgUf7AUiokXkTBEZr5Tany/oikspVUdE/i4is333BXExRESOEpEjRWSZiHQUkd1ee5RADEeXzBUi8rbWeofvjqDktNa7tdbztNYhEVEiki8idUSkrt+eoZQeFJHhIrLed0dQOoU/qAZJwYjG77rALK01SbiyU0pVF5HzRGSs776gdJRSM6Tgl/X7IjJaa73Oc5cQI6XUYSJyqIg847sviIuDRSRPRM5TSq1RSs1XSl3vu1OJxHB09M6Vgl/ak3x3BKWjte6klKoqImeLSBXf/UFsCucOnxKRG7XWIaWU7y6h9BqLSC0RyRaRFiLSRkS+UErN11p/5rVnCcKZcPSuEJEXNVeyJYXCoenXROROpVRn3/1BTAaKyAyt9WTfHUHc7Cr8339prXdprWeIyOsi0ttjnxKKJBwFpVQTEekuIi967griL11EWvruBGLSQ0TOLhy2XCMFF/M8qpQa4blfiN2Mwv+tNCc7DEdH53IR+V5rvch3RxA7pdQRUvA3P0UKlijdJCINRORHn/1CzK4UkaqB8v9E5G0RGeOlNyg1rfUipdQ3InK3UuomKfiBfKGIXOy3Z4lDEo5OXxF52HcnUGoZUnAVbUsRyRWRmSJymtZ6lddeISbBdaMiIkqpPSKyVWu9xVOXEB8XS8EPqQ0isk5E7tFaf+G3S4nDcLQrR0SmKaXuCz6otW6ntf7Tr2ul1FVKqc2FzwuVUR8RPed4aq0naa07a62ztNZ1tdbHa62/3tuY41nuFfn53Etr3T1so457ReTXwudVmuHNCuZPx1RrvVJrfYrWOlNr3VJr/ezeumT8jLJjFgAAnnAmDACAJyRhAAA8IQkDAOBJmV4d3SvlfCagPfks9FbctxPiePqTiOMpwjH1ic9ocon2eHImDACAJyRhAAA8IQkDAOAJSRgAAE9IwgAAeEISBgDAE5IwAACekIQBAPCEWxkCAMq/lFQTzh/V1amaffJTJj79igEmTvtiWuL7VUqcCQMA4AlJGAAAT0jCAAB4wpwwAKDcSWvWxCnPf7CeiZd0Hx3WuoqJNreycf0vEtK1uOJMGAAAT0jCAAB4wnA0Ko3UDtkmnjugjlO34JynTRwS9xasKWJvC/rU5hYmHvtYb6ddvTGT49JPoLJKa9ncxL/dXd+p+/MQtHXN8qNN3OCb9SbOj1/XEoYzYQAAPCEJAwDgCcPRSCppTRo75d/uPcDEr534rIm7ZoScdqHA79GQuHXB36r9ay80ccM7XnFaPTfhWBPnrVgZfafxJylVq5q46dfKqXuq0XcmTlX22MzZs9Npd+vJfU2cP2+hoHxS6fZq5jn/rGviJT0jDz+3/PyvTrlt/99MHNq9II69SzzOhAEA8IQkDACAJyRhAAA8qfRzwqtvOcopq8DqlKobbGFTO/d5B062F79XHT8lIX1DdBY/dKSJ5176pFMXXG4UXGoUCvv9+eHOWiaesr1lxPc6pMZSE5+budWpWzVhlok/6OgugcK+BeeBV75ul4J90OiVopqLiEj3WWeZWD3qLmnJWPRLqfuU1rypifOWLiv16+HP5o3obOIlPUdFbNd64pUmbtP3Z6cu/CqOioQzYQAAPCEJAwDgSbkcjl53vTtEvLlTronHnTQiru/VvsrUiHW7dZ6Ja6VUc+rWXb7DxKuGu/+Mj63pZeINF9Q0cd7yFTH3E5Gd38suWQnf7cpdbmR/cz65uZXT7rOTO5q4uOVF351+kYnPeOZppy64fOkD6VZ8p/EnC/9pb9Q+t9uTEdu1+eJqE7cdMM/EoR1LnXbuX0J05o90j9t7Jz1h4gtfuMWpa/rP72N4BywceoRb7vNUoGQ/oy0/c5chZfefbeJYjm15xZkwAACekIQBAPCEJAwAgCflZk54/ig7FzO39zCnLkOlB0tl1KPw93Xtn1ojELt1Lzb72sSXvdHdxJsuaeq0Y8lDKRx2sAmvq2fnZj/ceYDTLLjcaNbWhibOGbyf027RQ/YgZt9X3anLn2O3wQsuR0t/1j3wuYGJqpV3uNc1NPov84fh9JGdnfLXlzwcKNljsCzP3Y4yu59dChbK3VPqfuT2PMTE43q515x0DGypiNjtOcV+v487a6hTl6rs0jRnGdJVvzrtdKgi3BOp5DgTBgDAE5IwAACelJvh6KdPeNHE4cPA/93QxsTr9mTF9Pr/m2aHnJqOV8W0jM6KHvb3y0O9X3Xqgjspvdx8ookve7W7027ThfaOPyxfKqEpM03Y/9wBJk5dvdFp5i43WmOilXe4u2LNOd4uRTl11DVOXeocG2/oZ3fnytXTnHbB5VDNXvnd7Ud4/yFr73CHkvdPtUPQu7St6zvoVqdd9dwf49qP7X+zn9eDq7jfPdt1jolbvLXBqUvOwdHEqHf3EhN3qlLVqes153QTZ99rj0V+kg4/h+NMGAAAT0jCAAB4Um6Go4deeJ6J/9GlplO3/7t2V5z8De5wY7SyJfLOWLFoPd7Go5/r7dSted3u4HR97eUmDg5Ni4i07W+HUZvfw3B0rPRUOzQd7bBv1fXunjsjtzQ3cZW12526xUPslc4vXG6HrYM3hBARmZZjf9MWt+sWCvTP/jZi3dnzzjdx9XGRh59Vmv0KU9WqRWwXLv9gOx3xePvnI7brPu0qE+8/e27Urw/XTY0+j1i3daydlqu9YHJZdKdc4UwYAABPSMIAAHhCEgYAwJNyMyesp9k7ZNRzV36U+6UAoRnuXNHzj/cx8fVDng5vbrx6md0Z7K57Dot/xyqhXWe6/44b29k/8eA8cL2Z7rxv/1pLTdzlA3d50WEZ9nnBZUhTc9zfsP/oZ5c2pYp703GUTFb6bhPvCKvLPelQE9e9Z6mJ32j5aQneYVKRj34Xdkz3+0/Z7dCXbLZcZu+WdFzVX0x89IxznHa1X/qhzPpUHnEmDACAJyRhAAA8KTfD0UA8rLrQ3YVpzvF2OiC4pCgUdlvwYF1w+Dm8LrgM6fK3b3Datfyq8i2vKI2Ro053ytfdZm+e8GJLuwbwuu9PcdqNaWaPaZqE3T2llK4cf51TbjO5cg+VlsbmM8InEgrsfN+9yUqmXpy4TqSE/X2Uw124OBMGAMATkjAAAJ4wHB0HK+5y7x0b6rotquc1SLVDp3knHuLUpX05Lbw5YhC8mjn4m9N9vPi6/stPNPHyv9ubiTD8XDo7GocfA6uasvfxHdvsy7BaO8R46xp7NfxHE7o5rXIPtJ+vhSeNiqpP9X8u/c1dUODAOluLfLzahsjHPVY5p9pjv/4ae//pgxqsdtptO8/+XeWtXiPlAWfCAAB4QhIGAMATkjAAAJ5U+jnhtJbNnfLCfgea+KmLRkb1Gt2rursjparofts0Tss08cjnhzl1A5sdE9VrwNXwjSpO+fxGdhnMQTVXmfi6et877RoFbigf/tt00YPtTVztqylx6CVERLKf/cMpt8+9PqrntX7J3kktNG+RiVvkuXP0i/9zZFSvN3Dl0Sau+6p7LYYOb4yI0g5o4JRHtX0lUMqU0kqtXcvEZ01e4NRdmDXcxLVSIt9Nq+OIS03c+FzmhAEAqNRIwgAAeFJphqO3n3+4if/4i/3t8a9zXnfaXZS1KYZXL/1vmZ6fD3LK2fJTqV+zMqr2njtcnPOejacFjlP/bgOcdtvus7v7fHnwG07dMf+0uyb9Oq2JifNWrCxVXyu7/PmLnHKLOxdFaBn2vChfP21ndMuNfhrdxcT1c1l2FrP0dKfYNK10Q9DrBrpLP8+6dqKJ+9daFdY68hB00H5ZRe/i5RNnwgAAeEISBgDAE5IwAACeJNWcsOra0cS1R7jblX3U3N55JdolRO/usHMas3Y1jtjug4e6O+XUHLuw4Yp/2bvB/Hkew6qyJj1iXWWR1sT9N85bviJh76WnznTKmYEb9Zw/yb27z7jWH5n4oKvt0rGm/2ROuDxTxUwe5wVmluvMzymD3iQ/vc3drnfkloYmLu67L7V+PRMv/2tbE88c9FQce1dgy66qJt4/7q8eG86EAQDwhCQMAIAnFXo4+vch7iXs91xkl5ZcmrXBqVuWZ++sMXdPHRPf+NrVTrvqq+2yhgMnrjdx/m/zI/ajlkS+8feCvwd2kQkbklmSu93Ezd/bLpXRrjPtXXCCS4FERD743U4vHHjWnDLr05ZHmjrl0DN2eiG3za4y6wdK56qLJ0SsO3+hnXJInfhzxHaIXv7mLU75tRX2zkb9a9m1gkff8aPTrtt9i018QeYXce3TkD86OOWGN9klSnlxfafYcSYMAIAnJGEAADyp0MPRtbutc8rBIegev53h1OU+cYCJg7sqNZfIO+REuzNPuNDxXU18Vu0xgRr3N8/GUOBmA1Pcq3WTWfAq6Asf/NjEP21t7rQryyHo4Obw5/3HHcZMEW70XhGk7refU26TsTBi2/VPNzdxlpSPjfyTze7n7c1wch7ONfHDB0yP+3vlavtt3WFSPxNn/92dlsz7fXnc37u0OBMGAMATkjAAAJ6QhAEA8KRCzwnX6+cu62l9i70zTqvB7lxvmiwrkz6JiGzKtruyHF018u+c/rMuM3F9ibwEKtn8foldAhRcuvD49J5Ou1YS/7kj47CDneKpz39t+1TbnUsMBX6rps+P7m4tKHtbTmjllE+vbuf2t2t3V6yq63MFiVXzVbvk8Md/2x0Bj6taVOt9y9chEx/60yVOXZW37bLTli/Z7/7ysgypOJwJAwDgCUkYAABPKvRwdN5qd2lBq8HlY6nBhm5FD4LM2bPTKWc9VavIdsmu0Vd2o/f0m1NNfHOXL512Y248zcT1ZrvDiWlfTivytVM7ZDvlVT3qmzjzNPv38dXBLzjtgsuQQmG/TbM/vtbGQ74v8n3h3xVD3o9YtyTXPabpnxf994Oy1+7by02sZmWZuMXw2U47nW+Ho/ffNjfxHSsjnAkDAOAJSRgAAE9IwgAAeFKh54TLi5NnbXXK42o/GSjZrSmvmH2F067Ox1MT2a3yK7BF59EzzjHxlwe/4TS77s4nTBySkFM3ZN0hRb70GbVec8pdM+zzUgK/OcNfL/h7tO3b1zs1HR62W91VhCUPlVW91Mh3Intk9clhj2xObGcQUYenBzrl5g/abYR1nv2ExbptcEXDmTAAAJ6QhAEA8ITh6Dg4r+YMp1w9JdPE83PtTaSrj6hdZn2qKGpfs8fEQ953h5gfaGD/XXO1+7z79v/FxCGxleF3PAouN1qbv8vET204ymn36YijTdxmjLvbGkPQFd+eUOq+GyFh7m/ZxcRNxF3mp8MbVzKcCQMA4AlJGAAATxiOjtG6gXY4s0Gqe5Xzklx7lebFDww2cf2P3WFOiOQtX2HiX09v4tS1/m/RV0CLiMzpPtrEx824wMR/bKwZ8Tmth9qBZT11plNXTzg2yWxU8w+c8iGP/s3ErW79Ibw5UGY4EwYAwBOSMAAAnpCEAQDwhDnhKKmMDKd87nX2jj/bQnucut5TBpi46bPMNUYrb8VKp9zq0pURWor0ETtfXFMWBeLIKvtSiGR39+uXOuV2fR+zcbr7+ZWQu5QN8IUzYQAAPCEJAwDgCcPR0Qq5g5kvjT/BxB//2t2pa/omSx6Astbs/9ypn1v+78iIbVuxJA3lBGfCAAB4QhIGAMATkjAAAJ4wJxwlnesuQ2p+N3NKAIDS4UwYAABPSMIAAHiitGYfIQAAfOBMGAAAT0jCAAB4QhIGAMATkjAAAJ6QhAOUUloptUMpdX+U7fsppbYXPq91ovuHkonhePYsPJ4hpVTPRPcPJcPnM/nEcEyHFLbXSqmk2OeCJPxnnbXWd+8tKKVGKqXmFX4xXxlsqLUeo7XOLPMeoiTCj+eJSqmflVJblVKLlVL999ZprT8vPJ7LvPQU0eDzmXzCj2kXpdQ0pdTOwv/tsrdOa32viHT00ssEIQnv268iMlBEfvbdEZSOUipdRMaJyLMiUktELhSRx5RSnb12DKXB5zOJKKWqiMh7IvKyiNQRkbEi8l7h40mJJLwPWusntdZfiMhu331BqdUVkZoi8pIuMFVE5ohIB7/dQqz4fCad7lKwnfJQrXWO1nq4iCgROdFrrxKIJIxKQ2u9VkReE5GrlFKpSqkjRaSZiHzrt2cACnUUkRna3UVqhiTZEHRQUkxsAyXwmoiMFpFhheUBWuvlHvsDwMoUkS1hj20RkSwPfSkTnAmj0lBKtRORN0Skr4hUkYJf17crpU7z2jEAe22XgimjoJoiss1DX8oESRiVyUEiMk9rPUFrHdJazxORD0XkVM/9AlBgtoh0UkqpwGOdCh9PSiThfVBKVVFKVZWCiwPSlVJVlVL8u1VM00WkTeEyJaWUaiUifaTgCltUQHw+k85EEckXkZuUUhlKqRsKH//SX5cSiz/WfftURHaJyFEiMrIwPs5rjxATrfUiEfmriAwXka0iMklE3hGRMT77hVLh85lEtNZ7ROQsKZgy2iwFn9ezCh9PSiRhV46ITFNK3bf3Aa11d621CvtvooiIUuoqpdTmwueF/HQZxSjqeL6ptT5Ia52ltW6stb5Dax0SEVFK9Sg8ng2k4Nc4yhc+n8mnqGM6XWt9iNa6mtb6L1rr6XvrlFL3SsHIVY6IJMV9eLmfMAAAnnAmDACAJyRhAAA8KdPNOnqlnM/Ytyefhd5S+25VMhxPfxJxPEU4pj7xGU0u0R5PzoQBAPCEJAwAgCckYQAAPCEJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwhCQMAIAnJGEAADwhCQMA4AlJGAAAT8r0LkoV2ZLXOznlb49+2sSX9L3RqUv96ucy6ROAyBY9eoSJbz7lY6fuo4uPNHFoxtwy6xP24Qj7PbvkZvcmRPOPH2vi1hOvdOpaXfJLQruVSJwJAwDgCUkYAABPGI6Okl5WwynXO7aaiTe2zXDq9vuqTLqEOMo5rZuJN16z3amb3u2VqF7juhXHmvjbjzs7dS2fXWzivNVrYuki9iGtUUOnPOLM503cq9oup27s4b1NXG9GYvuF4q0ZdJSJH7jhOROfVG2H0y5X23jYYa87dcOlXZGvvfbGo5xyw1ft1EP+ho0l7msicCYMAIAnJGEAADxhODpKNVaoiHUHXPi7U85/JtG9QSxUehUTz3+sq1P34emPm7h1uju9EIry9Z9p/I19zjVfO3VdDu5r4sbnMhydCIuubeaUw4eg4Y/KsJ+pTRf8xan7+rZHTVxdVZHSWvF3OwQ99fqhTt2b1zc28fCh5zp1+z0zudTvHQvOhAEA8IQkDACAJyRhAAA8YU44DnblpTvl0s9qIBHmPdHFxPNPf8qpS5GqJg6Jlmj0X97dKY9uMili2+Fd7JKKR+sdb+LyskwiGTQ5eoXvLiCCxf+088Cz+44Iq43uG/OZzS1N/OxLpzl1jeR7E+fUs1dxpKtUp92lWatN3O3Ox5y6y+UWE5fl/DBnwgAAeEISBgDAE4ajo1TztNUR67a84+7Us5/8HqElEi24DEnEHYKe3Sc4DOYOU63O32ni48bd5tS1HLfHxBkL7PKi/PUbnHZd37jUxNO6vezU/byruYn1ntwIvUdJ7e5zmImHtXwirDZd4E9wWVKNDptK/PyPd2Y55XduP8nEjT78Prx5iWWHfVe8/vdHTHxy10G23bVTS/1exeFMGAAAT0jCAAB4QhIGAMAT5oSLkd/dXlY/vuOTTt0ve+ycYoNXZjl10W5ziPhbff2hTnn+6cF5QnvMxmxp6rT73zW9TNzmux8ivn5eMe+dkxN5DnL8Snuz8mrblhTzKiiJXfXsMT24CnPAPqk0N50s+pf9/vzt0PBlSUULLvtbd647J5yxMrq52eYf2ms4OjW70qmbduQYE4cvX2qRZpcp1pxbdn9LnAkDAOAJSRgAAE8Yji5Gfob9jZKp3Dvr5Gq7q1Jo27Yy6xOKN6D/e045Rezdrx7c0MHEk8/Idtqppb9E9fqpNWuaeMXVBzl1t3f6n4mn73EnJaqdzBC0T9/luOcbWcuLm1hALHJ6uncm++2y6Iagb151tInXnmaHgfM3rIqpH6lf/Wzipl+5dePmHWjiCzLXxfT68caZMAAAnpCEAQDwhOHoYiw9m98oFU1+2O/K4M0YPnqgu4mzlka+AlpS3Ksm84/vbOI+I74w8XW13bGu4ND3afPOCnvRlZHfDzFrd93sqNoNXdHLKVf5JLG7IFUWa286ysQDB7wb1XOCw88iIkuOt5/Z0M7Kd0MTsgwAAJ6QhAEA8IQkDACAJ8wJFyPrAJYeJZPqa/bsu5G4c8AiIh+/PCqq5529sLeJU87d6dTlR/UKKKmBDYLz8ipiu3kft3HKjeWPBPUouaV0bu+U/3OT3YGqR7Wd4c2N4E5YwWVIIomdB1ZdOzrl5uk/R2gpsjA3x8S1FpfdEjbOhAEA8IQkDACAJwxHI6ks2NXAfaDWUhM+9+JwE/9nbU+n2cTfW5v4k8OGi6uaibaEdpu424d/c1q1u9Uulwnt2BFtl1EGmr3rDj8zPRCbY19yh3OLG4IOmvruwSZutOH7uPapOPMGVHfKh2XoCC1FJuywO+pVe29KwvoUjjNhAAA8IQkDAOAJw9FhUqrae0oe0yjypvuj1h0fKG1PYI9QEnOu7+A+8M6PJjww1Q4rD2v4ndMspaEdIgsFhp/DnfDEYBNnP+QOq3Ef6bIR3KWpbXrwGFR12q3MDwyV5jEAHav11x5p4gF1Hg2rtTe2WZ2/y6m55Xe7a1zT/601caKPRFqLZiaedMrjYbWRP9vfbmwdKK2Pb6eKwZkwAACekIQBAPCEJAwAgCfMCYdJqV3LxE80/Dhiu0nf2hu6t5Ji7siDhMs5rZuJl1/k7nSTUswuSkGpKvB7VLuzuz1mn2Pihg+V3fIKFEhtsL9T7nrJTBPXTKka3tzoPu42E7dZwGc0VtvsFKtkpmREbPfIuhPc5x0bnFctuznWedcfaOLgdSDhNgWWG4qIrBnWysQ1mBMGACD5kYQBAPCE4egwec0b7LuRiDT9JDfBPUFQSqd2TvmAkStNPLrJsyYOibsjTqRlQ3eu6eaU/zflUBM/3WusUzem7csm7nuBHeLMfJMhzjJRv45THN3kkyKbbQ0bXsxawjlGWfrk80OdcguZXHZvruy0k06N7im3rTjVKdd4+8cILROLv1IAADwhCQMA4AlJGAAAT5gTDrP+7t1FPt577hlOucrEX00c+b4cKI31/e12eRPuecSpq+UsTYm8DOnW1UeY+OMv7ZxV9uPulqTZq+1dUx454VKn7uOXR5n4onvtsrUP3nTnKpEY+TWqRNVuZq57x5wDhrKcrCwd+J2/rUG3XHq4iede8GRUz/n+O3eLW19LTTkTBgDAE5IwAACeMBwd5umDXgmU7LXuq7bWdNo1zFtRRj2qPLZddIRTDg5B1wrbGWlOrl0i9viaXiaeN7Sj067Wu7+YuOVuu2TC3VfLlTrpV6fc7s3rTfzr+UNNPO6kG5x26Z/+VMyrIlZZj66Oqt2A6e40QmOZnYjuIIJmd811ymvHx/f10xo3MvGC65s6dT9eFry7U+RdvV7bZpegZj+/yanzNZjOmTAAAJ6QhAEA8KTSD0enNXeHNbKUvaIyVaWXdXcqtfWd3Kucg0PQ43bUdeqev+A0E4d++c3EWWFXOEbaMas4KdXcoe+Of1lq4ozA30QoLbqbQ6Dk0po0NnF25rKI7S5d2tPEza5e5dT5u1a3cjqm9kKn/G4bO72Uv2BxVK+R2r6NiRdcUd+pG3re8yY+qdqOsGdGHoIOGnv9mSZOmz0tquckGmfCAAB4QhIGAMATkjAAAJ5U+jnh3aPdcna6nQ/MD9zcPfNNd4kSEi8lsBPWHV9d4NRl/zI1ru+VWr+eiauPc+d632j5UaDEPHBZWNO7iYnf3/99py5V2XOHTbvtLlkpe9wlJyrd7rSlc/fEu4uVRpvRdonYkN5dnLp797NLAK+qudypS33ffn/O3NlYotGlxiQTX5oV3dK0cO/vsDvZ3fb5RU5dux/ssrVYrhdJBM6EAQDwhCQMAIAnlXI4OjW7lYlvbf5+xHYXL7E7MdV83c8NnyuT+jPcW2FsCu0y8dTeQ526bs8OMnH7//vdxPlr10V8/bRGDU28o3Mjp27QsNdMfFr1LU5dcNjqyc32b6faN3MjtkPiBKeJPmoX+PzOd9u1eXugjW/2szl/MshbvNTEE4Yf49QNGmL/XcN3tetbc6UtBOM42Knd6YUnN9ph8q//2s3E2T9NcdqVx88oZ8IAAHhCEgYAwBOSMAAAnlTKOeE9jWqZuEe1nIjt5r/R1sQNNDcIT7Ss1915u+NaDzbxrwOecOrm93nGxLNPsvdEGrTgwoiv/0p7e4es8Pmr4HKo8HmjW1fb7ffm3mhvBK62/SpIjKob7VFYlLfLqWuVVq3I5+wKmyesvppzjHir+9xkp/x/A3qY+Lr9Jjp17dPju+1v8HqMl4ad6tTVHxns16y4vm+i8VcKAIAnJGEAADyplMPRxbluxbHOhi8NAAAgAElEQVQmbvjaPBNzR5ayV3eu/Vd/ZnNLp65D1RUm7l7VDiV/1vGdYl6xasSaZ7Y0M/HjH/Zx6trcM93EajdD0GUh8y27JPCCAwY7db/8/SkT/3t9OxO/M/JEp12jEUwhJdqibrtNfGfri926Kw8w8cmn/GTiRw90p506vniDiVUxX7StXt1g4vq/TY7csILhTBgAAE9IwgAAeKK01vtuFSe9Us4vuzeD47PQW3G/84DP45nWvKmJF/yndsR2D/7lXRN/v621icdPONxp1+KuijW8lYjjKcJn1Kdk+4xWdtEeT86EAQDwhCQMAIAnJGEAADxhiRIqpLyly0zc4qJlEduNlODSJrsLUwupWHPAAJITZ8IAAHhCEgYAwBOSMAAAnpCEAQDwhCQMAIAnJGEAADwhCQMA4AlJGAAAT0jCAAB4UqZ3UQIAABZnwgAAeEISBgDAE5IwAACekIQDlFJaKbVDKXV/lO37KaW2Fz6vdaL7h5KJ4Xj2LDyeIaVUz0T3DyUTw/EcUtheK6W4Y1w5xHcuSbgonbXWd+8tKKVOV0rNKjzw3yulOuyt01qP0Vpn+ukmohR+PE9USv2slNqqlFqslOq/t05r/Xnh8Yx8b0T4Fn48uyilpimldhb+b5e9dVrre0Wko5deoiTMMVVK1VdKfaeU2qCU2qyUmqyUOnpvw2T8ziUJF0Mp1UZEXhGR60SktoiMF5H3+VVdMSml0kVknIg8KyK1RORCEXlMKdXZa8cQE6VUFRF5T0ReFpE6IjJWRN4rfBwV03YR+auI7CcFx/S/IjI+mb9zScLFO1lEvtFaf6u1zpOCP4hGInK8324hRnVFpKaIvKQLTBWROSLSofinoZzqLiJpIjJUa52jtR4uIkpETvTaK8RMa71baz1Pax2SgmOZLwXJuK7fniUOSbh4qvC/8PJBfrqD0tBarxWR10TkKqVUqlLqSBFpJiLf+u0ZYtRRRGZod7ODGcIQdIWnlJohIrtF5H0RGa21Xue5SwlDEi7eZyJyvFKqe+EQ110iUkVEqvvtFkrhNRH5PxHJEZFvRORurfVyv11CjDJFZEvYY1tEJMtDXxBHWutOUjBqdYkk+Y9kknAxtNZzReQKERkhIqtFpL6I/CYiK3z2C7FRSrUTkTdEpK8U/JjqKCK3K6VO89oxxGq7FHxRB9UUkW0e+oI4Kxyafk1E7kzm6zZIwvugtX5ba32Q1rqeiNwrBcOXUz13C7E5SETmaa0naK1DWut5IvKhiJzquV+IzWwR6aSUCk4ZdSp8HMkjXURa+u5EopCE90EpdUjh/OF+UnBV7fjCM2RUPNNFpE3hMiWllGolIn1E5FfP/UJsJkrBhTs3KaUylFI3FD7+pb8uoTSUUkcopY5RSlVRSlVTSt0hIg1E5EfffUsUkvC+DRORzSIyr/B/r/HbHcRKa71ICpY/DBeRrSIySUTeEZExPvuF2Git94jIWVIwvbBZCo7tWYWPo2LKEJEnRWSDiKwUkd4icprWepXXXiUQd1EKUErtloILdoZrre+Jov1VIvK4iFQVkQ5a68UJ7iJKIIbj2UMKknKGiPTWWn+V4C6iBGI4nveKyC1ScDxraK3zE9xFlBDfuSRhAAC8YTgaAABPSMIAAHhCEgYAwJMy3RS7V8r5TEB78lnoLbXvViXD8fQnEcdThGPqE5/R5BLt8eRMGAAAT0jCAAB4QhIGAMATkjAAAJ6QhAEA8IQkDACAJyRhAAA8IQkDAOAJSRgAAE9IwgAAeEISBgDAE5IwAACelOkNHICyltasiYk3H97IxKv77HHaDfjLJBMPqjPfqTvo26tMHFpaw8Sth/zqtAvt3Bm5HwceYOK81Wv21W0gqeT1OMTEGzpmOHW79rf3mNCtd5j4js6fOu361bKfm092uq8xeGQ/Ezd86PvSdbaMcSYMAIAnJGEAADxhOBpJZdXgo5zy3Ve/ZuKzM9dFfF5K4PdoSEJO3YxjxtjCMTbsvPtmp12zeyMPg2W8kW/ivOMiNsNeyt6Kdd2AI52qATe+a+L+tVbF9PIjtzQ08btnHGHi0NIVTjud605bIHpbLrP/rl/+Z7iJM5SbdkJS9C2PU8S9HW+utu16VHOnfr696VETH5V6q4kbP1j+h6Y5EwYAwBOSMAAAnjAcHSalc3sTz7ulmokv7/Kj0+7GulNM3OPRwU7dAUPL/xBIMkntkG3i4PCzSOQh6D/yc5zy73nVTZwv6U7doVXskGRqYJj016uHOe26bbXD0wc+6v4NHFN3kYknSM0i+1TppaSacPndh5t45nUjIj4lR9th/lV57jGtGhjN3D+1ulPXr6Yddu438W0TD9vU2mn3RZ+DTJy3dFnEfuDPtp613cTpyh7b8OHnZXm7THz3ijMivt6Pc1va16vhThN8e/TTJj7qLLtqYflj7lXUOsf9GykPOBMGAMATkjAAAJ6QhAEA8KRSzgmrDDtPsKb/IU7dj3faeb5tITvvcMTrtzntvu5i546Ov2yqUzdvaFy6iSjNvTPTxOFzwMFjeMJP15i4wbCqTrvUiT9HfP3119olMn0Gfm3iu+r/4rTLd6efHN9ubBUo/RG5YSW2cnC088B5Ju78qp2Hb3n7ZKddavs2Jp779yynbtaJz5g4uGTm5joL3Tf7wIafd2/hVOWv3xCxjxBpfs1KEw/8xK7Lm7XxAKddncBKv/z5iySSbNkYse7wZ/5m4vmn2/nhLrfe6LRr/ED5u16HM2EAADwhCQMA4EmlGY5OqWqHH+cO7WTihae7w15PbLZDWG8NOcXErd4MG+rKtsOLM1p1cer06XZtRNpOu4Qi7YtpJe02ovC/Y58OlNzflQN/t0seGp79W0yvX/9Ze+y/XGe3zLprxC9FNS/SvE/s31VjhqNFRESluV8/VY6Obnj3oP/ZIcY2YUPQQflzFth2fd26Y/vbMdCH7hhp4u5Vc512weHpL7IOdl+E4ehi5W/aZOLpo+yUTu1F7jKh/PmRp4Kilbqj6PPJjr3nOeUtD5T6reKOM2EAADwhCQMA4AlJGAAAT5J2TjilurtN3cpXm5l4YTe7POGxTW2cdhNuPN7EmV/9EPH1g5fSV9+01akbNHmiiUevsZfmb/liH51GTA6uYreZDN8Sb+p8u6wkW0o/h5c1y87nfrvbXeZUb3ZeeHNDq4hVlVZq08ZOeeohrxXZ7onNLZ1yu2fsXGN+eOMo1R9p55LHXXOoibs3jDzHjNjVG+3n37VP/V+d8ivSOEJLfzgTBgDAE5IwAACeJNVwdHAIeu6jBzl1wSHoRza2NfHXZ3Rw2qUuKfnl8suvdIe0e1SbYOKN+9nXe7F2J6dd/uYtJX4v/NkJs8418WcHvenUje0+2sT3i7uULFp5PeyuavvdZ6chWqa5x6/+rUtMvOM99zVU0fctr9SWXtgwYt12bZexvP7AKU5drd8iTxPFYvGVzU383Xj3bmlHZ4RMvKC/29+W99gdoXRe5KkIxF/Oqd2c8pW9JhbZ7t11XcMeKX/LAzkTBgDAE5IwAACeJNVw9B+XdjbxwjOedOo+3Gk3+f/6zI4mzluytNTvu6dW5LHGObvtEBbDz4mROcj+GT/9tjs10L/WfBPPf+owE3f472qn3dqT7FWTp98wyanrW9ve1KNhWvAuDe4dG15sOd7EfXq7G8fnVWM8WkQktV5dE99xxZsR2729zV7VXuuV+A4/h8ufbXdVumJCf6du4Rl2GmtOX/c75bR3Attw/TQrMZ2rxFJr1nTKay+239vXDnLne/rVXGHipXm7TLzhYfemG1UZjgYAAHuRhAEA8IQkDACAJxV6Tjitkbtk4PbBr5p4Zf5Op+7BeweauObi0s8xpbVsbuI+p/4YuSESLni3nJeGnerUDbjX1s09MzCnd6b7GimB36MhCbmVYXO/e92x5kinPP5ru/NSu5krnLprH7J3cJpwjzvXVZmowN3MLs1a57EnRas5N+wr8Yyi24mIzLvO/n/JvjpBHUpCKV3cZaGrutc28da2dqnXNUe712YMrvdVMa9qt6Tr+dEtJs4ePyXGXpYdzoQBAPCEJAwAgCcVejg6VM8d1ju3ht3Y/V/rD3fqar5a8iHo4E3HVw46zKm785o3THxRZvm77L0y2XWmPTbHXjs17q/f7/deJv7jlqYmTpmx0GnXeqf9G2P/pNL5alO7QGmzt34gemkHHuCUr5hkb9pwcvU1Jk4Xd4g4XaWW+r2Puc1ON2a/Ef/vgETiTBgAAE9IwgAAeFKhh6OLc0bN6U75g/43mzh9Z+TdizaeZndb+eCop0zcKs0dQnl3h72ir/X71zl1wV12pm5sFqhZVXynEbWNV9krky+49VMTD6ozP6xldL8zg0NiHZ50d7tqcv/3gZIdGg2/hro4KaokrZPX4qubR9Vu1uv2CtoG8n0xLVFe6Dru9ODZNTYGSlUS+t7ODVJCsd5l2g/OhAEA8IQkDACAJyRhAAA8qdBzwqGZ85xy9pv2MvX5Fzzl1E25170DSjQ+2VXPxGeN/qtT1/ShaSZu13ar+8TALjsLpto54ZbMCccsrVkTp3zPXWNNfGr1bSYO3+1qY769OfwZM+wxfPGgF5x2rdPtrlhpu0vV1SKFNL93RUR2N9vjuwtIlNXuUs3Dp11i4q77rzTxN18e7LSrtlZJUXY1cK/d+de5r5v43Mz1Tl3vuyaa+CPpbuKs1xN7B6544JsBAABPSMIAAHhSoYejRbvDFa3/ZoceDpt7vVMX6r1JirJ5XZZTbv6Ojat8YndeaRK2TCL4znrGXKfu3+sPMvFlJ9tNyL+/PbGX6Seb1LatTfzghJedurbpdknRsjw75Nz75cFOu9ZP/W7iuivt8qU+L7l/H3NPHG3bnRw2bfB4YEefGJc/jHn1FBM3ZskNklD+Jvc7dr8zbDl4O5MWMlli8dITdhfEJ56v7tR9ebDdwXDSNW1sxZthu3GVw+VLnAkDAOAJSRgAAE9IwgAAeFKx54SLUf/ZsHmHZ4tut38c3iu1Xl2n3LW6nZuetrNFHN6hclpwb6aJg3PAIiKf77Jz+f+8/yYTN3/ePe6R7mbU+nJ3W9NzJ51m4gkd33Lqjhhotzzdf0Rs87mNH2AeeF9W5+80cc1l5f8+VDUWco1HWcpbbe/ElHmKW3fr1GNM/FG7d018xDU3OO3+lBfKAc6EAQDwhCQMAIAnSTscXZZ0I3dQ+7Tq20188zf2bj/Z8lOZ9SkZvHDEcxHrHr75chPX/bD0Q0yLPmlpC+4Illw9cLyJ3x9RT5AYWSl2yiGnpo2rJfh9U9vbJS2XXTMh6uc1G7vYxOV/8Dw+UuvUccp6j90BLbRjR1l3x/jk664mfvwiO/Vz9vVfOe2+ebZqmfUpWpwJAwDgCUkYAABPGI6Og5W96kasS1ufXoY9SS6pgX3JUsJ+L2ZsyAlvXirNX7BDiy/3dW8WcXS1hSb+sH62ifPXb4hrHyqDrNmBK4pPdusylb2JxpE3293q5ryY2D41esHukHZLnQUR27Uf6+6y1vKPqRFaJpe0Jo1N3OG9lU7dB+/Z6bamQxK7AkBl2L+PZYMPcepu7/1uePOCPlVZH/ZI4yLb+cSZMAAAnpCEAQDwhCQMAIAnzAnHQU4dve9GKLGXNxxl4q4Nv3Xqlv7Nxi0f7GDi0C+/xfReOs/eXWVLvnuHlvZV7G/VdWfbOeF6o6JfGrXtoiNMXBFuNJ4oTV5fagu3RG53cHV73505ckDc+7H4P3Yu881GjwVqMpx2o7bY6wNaP77QqcvPqxwLk7Yc1sjE/2nwvlN319XfmfiQ+n9z6tqO3lri91p8fm0T59YJOXX39XzbxBdkuvPPKaJMHHzWU/ed57SrJeXvs8eZMAAAnpCEAQDwhOFolFuffv4XW+jrDkfPOGaMiVe9Z5crPbquh9Pu42+6SjTGnTPUxOE3i5ieY3+r7vfKryZ2B8uKd94/PjXxhNdrluCZyUUHdlUatqm1U3dzHTvce3HWMhPf/2Jvp13bR+yNHkIz5kb1vtvPP9wpT7/scRNXCyyNCg4/i4i8f66dEsn/I/LypWRWY+UuE/97/UFO3T/qzzLxvHOecupSzgkOEQeXGyqnXbDOeX6U7URE1gVu/nH0e7eaOPtt90Yt5XHikDNhAAA8IQkDAOAJSRgAAE+YE06AVGV/29SZ7bEjFVzroYtM/OOF7vafh2fkmrhxmr3PzqNhS5kevdAtR5Ii9vVDYbO9H2/rZOt27pRYjJpztImbysyYXiMZ5G/eYuIv+rjzi/KBDYPzwwt6jHaavXSYXbL039fdJShBl57zpY1rPerUVVPVw5uLiMgTL5/plBvPSexWjBXCDzNM+PUtRzpVJ/3dzuv/r90bTl1wG9Lw+d0gd3mRnbV9ZZt7d7rzMu32oh0/GejUNRtnX6PNhz+auDzOAYfjTBgAAE9IwgAAeMJwdALkazucWWfOdo89qdjy164z8X9OOdepmzdwPxP37/GFiQfVjW3HrH7LTjDx1AnuMGnLMcsCpRUSi6bnV94h6Ejyli5zyq8OC9xW6eZAWMfdqeryrDU2vmZElO/mDj+/sLWhid8573gTN57zoyCytC+muQ/Yj56ccfrNTtWqi/eYeMqxdvnSefMuctqt/8De2UgFZoIavuIuPxvb2U4VZH/5U9R9Lu84EwYAwBOSMAAAnjAcnQDBq6MRH/nzFznl1oNs+UupEYi7xfgOdrP5puJeEVs5tun3L3hDjE9fqG/iz5t3cdrNvcFeNXvMYXb64dspHSSSdiM3OeXQ/CUm1rnzSt5Z/EnV8VOccsvxNr5I7M5jaeJOQxwQVt4rP6yc9uXGUvWvvCJbAADgCUkYAABPSMIAAHjCnHACLMq1y5JSN9sdlsLnOAAUTefa5S35CxY7dW1utuW1wceLuWE7nz2UV5wJAwDgCUkYAABPGI6Og+b/mOyUB/7jmEDJXVoDAMBenAkDAOAJSRgAAE9IwgAAeEISBgDAE5IwAACekIQBAPBEaa199wEAgEqJM2EAADwhCQMA4AlJGAAAT0jCAAB4QhIOUEpppdQOpdT9Ubbvp5TaXvi81onuH0qG45lcOJ7JJ4ZjOqSwvVZKJcW9D7g6OkAppUWkjdZ6YeCxkSJyvIi0EZG/aq1fiOZ58C/8uCilskXkYRE5SkRSRWSqiNyktZ5X3PNQPhRxPI8VkY/DmtUQkfO01u9Eeh7KjwjfuV1EZIyItBeROSLST2v9S6C+uYgsEZF0rXVemXY4ATgT3rdfRWSgiPzsuyMotdoi8r6ItBWRBiIyRUTe89ojxExr/Y3WOnPvfyLSR0S2i8gnnruGGCmlqkjBZ/JlEakjImNF5L3Cx5MSSXgftNZPaq2/EJHdvvuC0tFaT9Faj9Fab9Ra54rI4yLSVilVz3ffEBdXiMjbWusdvjuCmHWXglvsDtVa52ith4uIEpETvfYqgUjCqMyOE5E1WusNvjuC0lFKVReR86TgzAkVV0cRmaHdedIZhY8nJZIwKiWlVGMReVJEbvHdF8TFuSKyXkQm+e4ISiVTRLaEPbZFRLI89KVMkIRR6Sil9hORT0XkKa31a777g7i4QkRe1FxpWtFtF5GaYY/VFJFtHvpSJkjCqFSUUnWkIAG/r7WOalkEyjelVBMpmEt80XNXUHqzRaSTUkoFHutU+HhSIgnvg1KqilKqqhRcHJCulKqqlOLfrQJSStUUkQki8p3W+k7f/UHcXC4i32utF/nuCEptoojki8hNSqkMpdQNhY9/6a9LiUUy2bdPRWSXFKwtHVkYH+e1R4jV2SLSTUSuKtzEYe9/TX13DKXSV7ggKylorfeIyFlScEw3i8hfReSswseTEknYlSMi05RS9+19QGvdXWutwv6bKCKilLpKKbW58HkhP11GMZzjqbUeW3j8agTXl2qtl4lwPCuAP30+RUS01u201mPCG3M8K4SivnOna60P0VpX01r/RWs9fW+dUupeKdi7IUdEkmL+nx2zAADwhDNhAAA8IQkDAOBJmd6FolfK+Yx9e/JZ6C2171Ylw/H0JxHHU4Rj6hOf0eQS7fHkTBgAAE9IwgAAeEISBgDAE5IwAACekIQBAPCEJAwAgCckYQAAPCEJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwpEzvolTRLBj7FxPP6znKqTvxhoEmrj7uxzLrEwBUBqkd2zrlpefUM/GhvWc5dS82+9rEuTo/qtfvcf0Ap1zt3Skl7WJccCYMAIAnJGEAADxhOLo42t6TOSQhp2plDxu3GVdWHcJeaS2amXj52Y1MvC07z2nXNnulice3fd/E2R9c57RrPMH+Hq05fY1Tp7fvNHH+H3+YWKW5H59VNx1m4rxqbn+bPjLNvl5OjgD4s62XHGHi0+6c6NSNqzcz4vNytf38hn9XR/L00GFOefC8vibOn7MgqteIB86EAQDwhCQMAIAnDEfHqFX7VSZWGRlOHcON8bdm0FFO+afBT5g42uGnYKv5fZ5x6/pEfo03th1o4uf+draJVx3rfnxmXuEObwWdPvEaE6vvftlXV4GklVK1qlNe9M+uJp59+QgTR/u5jlV2ehWnPOfmOrbuuvDWicOZMAAAnpCEAQDwhCQMAIAnzAnH6KN275r4zMxeTl0+c8Jxkdq6hYnH3vx4WG3J/3THbd/fxOdmro/6eRdmrbbx6KdMnBL2GzY4gzU9x61L3bK7yHaVzdqb7Nz+1kN3F9MysdIz7FK2Wcc8H7Fdn0aHlEV3kp+yyz2Dc8AiIjMvHx4olf68sMObN0as++2CJyLWPXjCWyZ+/rA+tmJK5KVR8cCZMAAAnpCEAQDwhOFolFuretulQe2rRP69eOLMC01c476aEdulr95s4jENazt1OfXscoWBD73l1J2duW7fnRWRWXu0iQffOtCpqz6Lm3yIiOw4wu4+Nuf4URHbBYf6Y12qEu1rBGte3tokpvfCn4WOtcPOi/vbx387cXgRrf/s7e0HOOV/fGuXBzZ53/0+qPaevflCa/nBxKprR/dFL4j8fsHP+fCWNUycleD7OnAmDACAJyRhAAA8IQkDAOAJc8Iot465fFrEutX5u0y8dmYDE6eeGvn1Gvxk533XHprqvldPuwwh2jngcB9s7WLi6uOYAy5Km4FLTHxO1tlO3ZIrm5o4p46dqVVaYhKqv8fEc3o+G7Fdu4/s/H372xeG1W6K7c0ro8AyJJHweeCRUb3E6fPOMHHonv2cuuzvfoq9b+UYZ8IAAHhCEgYAwBOGo1FuffhTZxM/dPo3Tl3TtEwTz7lkhETlKhumK3c4OlfnB0rub9P1gaHvY9+6zcQTz3/EaXdXfTuk3f2C6526zDd/EIjkb95iC8FYRJrctyKu77X9AnuDeOnp1i3MtTtmtX94o+3fJoafSyJ4R6TwnbCiXYr0Y066ifWJK02sZGVRzZMOZ8IAAHhCEgYAwBOGo1FuZQ+wW9X8pV4/p27m0S+YOJYdlXLDrrh9f4e9ofewJT2cupRh9U3c6iM7rHxsjVucdnNPf9LEq3rlO3XZb5a4iyil1X32RKwbssJu0J8/f1FZdCcp6fatTOzeiCGy9l9c65RbjbSf3xT5JT4dq0A4EwYAwBOSMAAAnpCEAQDwhDlhVAgth7jze907Xh+hZWxq/7TGxNUWLwmrDS/v28HZy51yTiydQqks6DHaxKGw841pU9qYuLVsKLM+JZuVPWqZOKWYc7pxO+qauM2IXLdyykwpK8E+/nmZoo21u/lXQnEmDACAJyRhAAA8YTgaFUL+7HlOOXN2fF8/b99N/qRtduQdfWbOd28Ony1rIrREooREB2J3GVusN4Wo7NKaNHbKp1wy2cTFLRW848sLTZw9ZUrEdvG24h63HOxj+DLFK5babdXqfPibid3FhvHHmTAAAJ6QhAEA8ITh6OIExqzCr/wLv7IOlUNuz0NMPKGte4/UyYGN6Ns+tdOpY/Qz8XadeVjYI5HvR51f116hu/hVex/oQ5otc9oNOvAz+xxxL5m95rkbTNzk39+XpKsV1sZj3eHofzcYF7Ftr1kXmLj97XNNnOjh3aVvdDLxc11eiPp5i55pZ+LaWycX0zK+OBMGAMATkjAAAJ6QhAEA8IQ54eIEtk0Jv/w+eHn7nAdaOXXZ124UJI+UrCwTPzDSzgOHXxfw9XY7p6Snx3kNVRJKbbC/U952VAsT76przw9Szlkf1euN7Tg07JGMiG3nnvRMVK/Z7/deJp72SQenrvlj9o4/Jb+PV8W04Yyd+25UaPmKeibO3lryXedidXunT018aEbkGeh+y05wyvU+WWjiRM9bB3EmDACAJyRhAAA8YTg6HqpUlsGoyiG1Xl2nvP1Vu0l914zIO+48N+l4E7eRHxPTuQou96RDTZx1z1KnblzLESYOLgksbicmV/q+mxQKDjP/cUvTyA1/mGHCpuIuQ6qMn/q7unzilIu7aUN2v58S3R1j68d2SrBvzeDStMj9++25jk653h9ltywpiDNhAAA8IQkDAOAJSRgAAE+YEwbCLO/Xzin/dNCwItv9e30np9z+8bUmjuWuTJXB76far5wJLSc4da9sa2TizfnVTfzeqs5Ou3VfNZKiDO/3rFPuUc0uNOn288VOXd0+8wOlzcV3Gka+ds/bop+vL73U2vbajIXPNHPqZnd6Pqo+dXjzRhO3HuVnDjgcZ8IAAHhCEgYAwBOGo1EpBXfBEhFZ+0pDE7/T+eGw1lVMNHyTHaqe8NCxTqtai3+IXweTVO05dhe67I+uc+raD7ZDxPmbt5i4ivzutGscVt7r10vcIcrjqi6IuZ/wL3jHMhGRBvfZ4zmu6Ziw1kWfT36+y/2ctx1ldzMsy12xisOZMAAAnpCEAQDwhOFolFtrBh1l4io93U38H+3wpolDOrrfkvcvPc3EQ1q869QFd8IKDj+H++pCu+NTraL+eqQAAAP+SURBVNkMP5dU/ZGTA7Fbl8jhwfSX6+67EeJqw9VHmrje6OiuRJ7/vB2CbtZog1M3qukXJe7DjR9f4ZTb/Fb+drLjTBgAAE9IwgAAeEISBgDAE+aEUW5su/AIp/zT4Ccitk1XqSbO1blRvf5H7ew8cPD5Ba9h4y2h3U5dj0cGm/iA2e6ddOBXaoP9TdwwveilSyIiabsr4z2P4u+hGSc55b7HPB+hpUiHfrNN/NOB9vqO/hd95LS7vvYiE6erX0ycq8OvEoh8zhj8PGePvcHEbf5ePnbFKg5nwgAAeEISBgDAE4ajw6g0+0+SUWOPx55UPqt7ubc9KG4j9uDwcSybyAefH/4a/7emh1PX6NM/TFxedtlBgW1HtTDx2ZkfhtVyjhFvjUemO+XJ3eww8OEZ7rSQs6TousjLi4Kf3mg/18Gd60RERo23w+Qt//mzicM+5uUSf6UAAHhCEgYAwBOSMAAAnjAnHCalRVMT/3LUcxHbBZexNPyIf8ZYpdaz2wlefMgUjz2xHm/4jVP+anymiZ84pruJ89asLasuIQopYecUwc9o2nZm8+Mh7YtpTvm2IQNM/M0Dw+P6XivycpzyQ2t7mXj5lU2cuha/2aVIFWEeOIgzYQAAPCEJAwDgCeOo4TZuNuHBL95k4kOPm+s0W/FwGxNnvlv+7sxRUeR2sDdiv3f/CXF//VN+O8/Eayc1shXKbXfnpfauTBdmrXbqTqi23cRPZES+wxL8Cl/S8uKWg02c/vm08OaIg/qf2N2uuja52ambPmBYqV777GG3O+UDHwvuVje/VK9dnnAmDACAJyRhAAA8YTg6TP6GjSZuEdj8e0NYu2pSPq7krejSV9vh/2OmX+rUfdv1lYjPW52/y8S9XrI3WGg9coXTLmOVHVpukht5g//Xn+lq4jeqH+nUbT2koYmztibPMFiyGzXnaBM3lZkee5K88teuM3GTf69z6s74d7dSvfaBUjlulsKZMAAAnpCEAQDwhCQMAIAnzAnDq/yFS0xct49bd4ZEN6fUXOzcfV4x7Yrtxx9/RKyr/vty2y7G10dirOwZuS7zo8zIlUA5wZkwAACekIQBAPCE4WgAFVbKbrv12aMbDnLq6j4/Obw5UO5wJgwAgCckYQAAPCEJAwDgCXPCACqsVrf+YOJJUs1jT4DYcCYMAIAnJGEAADxRWmvffQAAoFLiTBgAAE9IwgAAeEISBgDAE5IwAACekIQBAPCEJAwAgCckYQAAPCEJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwhCQMAIAnJGEAADwhCQMA4AlJGAAAT0jCAAB4QhIGAMATkjAAAJ6QhAEA8IQkDACAJyRhAAA8+X8tgscturQpyQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 8))\n",
    "\n",
    "for idx in range(16):\n",
    "    plt.subplot(4, 4, idx+1 )\n",
    "    plt.axis('off')\n",
    "    plt.title('[{}]'.format(np.argmax(mnist.train.labels[idx])))\n",
    "    plt.imshow(mnist.train.images[idx].reshape((28, 28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，定义用于训练的网络，首先定义网络的输入。\n",
    "\n",
    "这里我们直接使用上面的数据作为输入，所以定义两个placeholder分别用于图像和lable数据，另外，定义一个bool类型的变量用于标识当前网络是否正在训练。\n",
    "\n",
    "为了让网络更高效的运行，多个数据会被组织成一个batch送入网络，两个placeholder的第一个维度就是batchsize，因为我们这里还没有确定batchsize，所以第一个维度留空。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.706044Z",
     "start_time": "2018-06-01T06:32:51.698913Z"
    }
   },
   "outputs": [],
   "source": [
    "x = tf.placeholder(\"float\", [None, 784], name='x')\n",
    "y = tf.placeholder(\"float\", [None, 10], name='y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为我们输入的是图片展开后的一维向量，所以第一步就需要先把一维向量还原为二维的图片。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.719298Z",
     "start_time": "2018-06-01T06:32:51.707730Z"
    }
   },
   "outputs": [],
   "source": [
    "x_image = tf.reshape(x, [-1, 28, 28, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:07:36.530623Z",
     "start_time": "2018-06-01T06:07:36.522665Z"
    }
   },
   "source": [
    "接下来，我们定义第一个卷积层，使用6个5X5的卷积核对输入数据进行卷积，\n",
    "padding方式选择valid，所以输出数据的宽高变为24x24,但是深度已经从原来的1变成了6。\n",
    "本层卷积的激活函数为relu。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.764292Z",
     "start_time": "2018-06-01T06:32:51.721295Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('conv1'):\n",
    "    C1 = tf.contrib.slim.conv2d(\n",
    "        x_image, 6, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来进行stride为2的最大池化，池化后，输出深度不变，但是长宽减半，所以输出变成了12x12,深度6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.774112Z",
     "start_time": "2018-06-01T06:32:51.766784Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('pool1'):\n",
    "    S2 = tf.contrib.slim.max_pool2d(C1, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:10:16.678485Z",
     "start_time": "2018-06-01T06:10:16.671472Z"
    }
   },
   "source": [
    "接下来，我们定义第二个卷积层，使用16个5X5的卷积核对输入数据进行卷积，\n",
    "padding方式还是选择valid，输出8x8,深度为16，本层卷积的激活函数为relu。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.805912Z",
     "start_time": "2018-06-01T06:32:51.776959Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('conv2'):\n",
    "    C3 = tf.contrib.slim.conv2d(\n",
    "        S2, 16, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再进行一次stride为2的最大池化，输出为4x4,深度16。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.814748Z",
     "start_time": "2018-06-01T06:32:51.807560Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('pool2'):\n",
    "    S4 = tf.contrib.slim.max_pool2d(C3, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "池化后的数据是3维的，这里做一个拉平的操作，将3维数据展开到1维，然后送入两层全连接，全连接隐层中神经元个数分别为120，84。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.856740Z",
     "start_time": "2018-06-01T06:32:51.817287Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\layers\\python\\layers\\layers.py:1624: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use keras.layers.flatten instead.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('fc1'):\n",
    "    S4_flat = tf.contrib.slim.flatten(S4)\n",
    "    C5 = tf.contrib.slim.fully_connected(\n",
    "        S4_flat, 120, activation_fn=tf.nn.relu)\n",
    "\n",
    "with tf.name_scope('fc2'):\n",
    "    F6 = tf.contrib.slim.fully_connected(C5, 84, activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:17:39.880958Z",
     "start_time": "2018-06-01T06:17:39.869797Z"
    }
   },
   "source": [
    "###### 对特征添加一个0.6的dropout，以40%的概率丢弃特征中的某些数据，\n",
    "这样可以提高网络的推广能力，减少过拟合的可能性。\n",
    "\n",
    "需要注意的是，dropout仅在训练的时候使用，验证的时候，需要关闭dropout，\n",
    "所以验证时候的keep_prob是1.0。\n",
    "\n",
    "dropout的输出最终送入一个隐层为10的全连接层，这个全连接层即为最后的分类器。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.913419Z",
     "start_time": "2018-06-01T06:32:51.860766Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-14-413a88f1eb86>:3: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('dropout'):\n",
    "    keep_prob = tf.placeholder(name='keep_prob', dtype=tf.float32)\n",
    "    F6_drop = tf.nn.dropout(F6, keep_prob)\n",
    "\n",
    "with tf.name_scope('fc3'):\n",
    "    logits = tf.contrib.slim.fully_connected(F6_drop, 10, activation_fn=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来定义loss和用于优化网络的优化器。loss计算使用了sparse_softmax_cross_entropy_with_logits,\n",
    "这样做的好处是labels可以不用手动做one_hot省了一些麻烦。这里使用了sgd优化器，学习率为0.3。\n",
    "\n",
    ">试试看，增大减小学习率，换个优化器再进行训练会发生什么。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.084738Z",
     "start_time": "2018-06-01T06:32:51.915376Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-15-e1c7953954e2>:2: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "\n",
      "Future major versions of TensorFlow will allow gradients to flow\n",
      "into the labels input on backprop by default.\n",
      "\n",
      "See `tf.nn.softmax_cross_entropy_with_logits_v2`.\n",
      "\n",
      "Conv/weights:0\n",
      "INFO:tensorflow:Summary name Conv/weights:0 is illegal; using Conv/weights_0 instead.\n",
      "Conv/biases:0\n",
      "INFO:tensorflow:Summary name Conv/biases:0 is illegal; using Conv/biases_0 instead.\n",
      "Conv_1/weights:0\n",
      "INFO:tensorflow:Summary name Conv_1/weights:0 is illegal; using Conv_1/weights_0 instead.\n",
      "Conv_1/biases:0\n",
      "INFO:tensorflow:Summary name Conv_1/biases:0 is illegal; using Conv_1/biases_0 instead.\n",
      "fully_connected/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected/weights:0 is illegal; using fully_connected/weights_0 instead.\n",
      "fully_connected/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected/biases:0 is illegal; using fully_connected/biases_0 instead.\n",
      "fully_connected_1/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/weights:0 is illegal; using fully_connected_1/weights_0 instead.\n",
      "fully_connected_1/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/biases:0 is illegal; using fully_connected_1/biases_0 instead.\n",
      "fully_connected_2/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/weights:0 is illegal; using fully_connected_2/weights_0 instead.\n",
      "fully_connected_2/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/biases:0 is illegal; using fully_connected_2/biases_0 instead.\n"
     ]
    }
   ],
   "source": [
    "cross_entropy_loss = tf.reduce_mean(\n",
    "    tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "\n",
    "l2_loss = tf.add_n([\n",
    "    tf.nn.l2_loss(w)\n",
    "    for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES)\n",
    "])\n",
    "\n",
    "for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES):\n",
    "    print(w.name)\n",
    "    tf.summary.histogram(w.name, w)\n",
    "    \n",
    "total_loss = cross_entropy_loss + 7e-5 * l2_loss\n",
    "tf.summary.scalar('cross_entropy_loss', cross_entropy_loss)\n",
    "tf.summary.scalar('l2_loss', l2_loss)\n",
    "tf.summary.scalar('total_loss', total_loss)\n",
    "learning_rate = tf.placeholder(tf.float32)\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(total_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:25:56.449132Z",
     "start_time": "2018-06-01T06:25:56.438340Z"
    }
   },
   "source": [
    "需要注意的是，上面的网络，最后输出的是未经softmax的原始logits，而不是概率分布，\n",
    "要想看到概率分布，还需要做一下softmax。\n",
    "\n",
    "将输出的结果与正确结果进行对比，即可得到我们的网络输出结果的准确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:39:50.010829Z",
     "start_time": "2018-06-01T06:39:49.997501Z"
    }
   },
   "outputs": [],
   "source": [
    "pred = tf.nn.softmax(logits)\n",
    "correct_pred = tf.equal(tf.argmax(y, 1), tf.argmax(pred, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "saver用于保存或恢复训练的模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.127795Z",
     "start_time": "2018-06-01T06:32:52.103115Z"
    }
   },
   "outputs": [],
   "source": [
    "batch_size = 100\n",
    "trainig_step = 3000\n",
    "\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上定义的所有操作，均为计算图，也就是仅仅是定义了网络的结构，实际需要运行的话，还需要创建一个session，并将数据填入网络中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:35:22.270272Z",
     "start_time": "2018-06-01T06:33:18.829198Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 100 training steps, the loss is 0.494236, the validation accuracy is 0.9004\n",
      "after 200 training steps, the loss is 0.26048, the validation accuracy is 0.934\n",
      "after 300 training steps, the loss is 0.19417, the validation accuracy is 0.9546\n",
      "after 400 training steps, the loss is 0.127495, the validation accuracy is 0.9664\n",
      "after 500 training steps, the loss is 0.153523, the validation accuracy is 0.9596\n",
      "after 600 training steps, the loss is 0.162495, the validation accuracy is 0.9694\n",
      "WARNING:tensorflow:From d:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\training\\saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n",
      "after 700 training steps, the loss is 0.145152, the validation accuracy is 0.975\n",
      "after 800 training steps, the loss is 0.151206, the validation accuracy is 0.973\n",
      "after 900 training steps, the loss is 0.0171238, the validation accuracy is 0.9762\n",
      "after 1000 training steps, the loss is 0.167053, the validation accuracy is 0.9776\n",
      "after 1100 training steps, the loss is 0.0859434, the validation accuracy is 0.9806\n",
      "after 1200 training steps, the loss is 0.151023, the validation accuracy is 0.983\n",
      "after 1300 training steps, the loss is 0.0357339, the validation accuracy is 0.982\n",
      "after 1400 training steps, the loss is 0.018307, the validation accuracy is 0.9852\n",
      "after 1500 training steps, the loss is 0.0160653, the validation accuracy is 0.9832\n",
      "after 1600 training steps, the loss is 0.0578578, the validation accuracy is 0.9838\n",
      "after 1700 training steps, the loss is 0.145009, the validation accuracy is 0.9846\n",
      "after 1800 training steps, the loss is 0.223466, the validation accuracy is 0.98\n",
      "after 1900 training steps, the loss is 0.12996, the validation accuracy is 0.9828\n",
      "after 2000 training steps, the loss is 0.0424626, the validation accuracy is 0.9836\n",
      "after 2100 training steps, the loss is 0.0449289, the validation accuracy is 0.9852\n",
      "after 2200 training steps, the loss is 0.00906383, the validation accuracy is 0.9878\n",
      "after 2300 training steps, the loss is 0.0191946, the validation accuracy is 0.9866\n",
      "after 2400 training steps, the loss is 0.0539525, the validation accuracy is 0.9878\n",
      "after 2500 training steps, the loss is 0.0233668, the validation accuracy is 0.9884\n",
      "after 2600 training steps, the loss is 0.0692054, the validation accuracy is 0.9828\n",
      "after 2700 training steps, the loss is 0.046307, the validation accuracy is 0.9874\n",
      "after 2800 training steps, the loss is 0.0303163, the validation accuracy is 0.9874\n",
      "after 2900 training steps, the loss is 0.0651424, the validation accuracy is 0.9854\n",
      "the training is finish!\n",
      "the test accuarcy is: 0.988\n"
     ]
    }
   ],
   "source": [
    "merged = tf.summary.merge_all()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "\n",
    "    writer = tf.summary.FileWriter(\"logs/\", sess.graph)\n",
    "\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    #定义验证集与测试集\n",
    "    validate_data = {\n",
    "        x: mnist.validation.images,\n",
    "        y: mnist.validation.labels,\n",
    "        keep_prob: 1.0\n",
    "    }\n",
    "    test_data = {x: mnist.test.images, y: mnist.test.labels, keep_prob: 1.0}\n",
    "    lr=1\n",
    "    for i in range(trainig_step):\n",
    "        if trainig_step > 1000:\n",
    "            lr=0.3\n",
    "        if trainig_step > 2000:\n",
    "            lr = 0.1\n",
    "        xs, ys = mnist.train.next_batch(batch_size)\n",
    "        _, loss, rs = sess.run(\n",
    "            [optimizer, cross_entropy_loss, merged],\n",
    "            feed_dict={\n",
    "                x: xs,\n",
    "                y: ys,\n",
    "                keep_prob: 0.6,\n",
    "                learning_rate: lr\n",
    "            })\n",
    "        writer.add_summary(rs, i)\n",
    "\n",
    "        #每100次训练打印一次损失值与验证准确率\n",
    "        if i > 0 and i % 100 == 0:\n",
    "            validate_accuracy = sess.run(accuracy, feed_dict=validate_data)\n",
    "            print(\n",
    "                \"after %d training steps, the loss is %g, the validation accuracy is %g\"\n",
    "                % (i, loss, validate_accuracy))\n",
    "            saver.save(sess, './model/model.ckpt', global_step=i)\n",
    "\n",
    "    print(\"the training is finish!\")\n",
    "    #最终的测试准确率\n",
    "    acc = sess.run(accuracy, feed_dict=test_data)\n",
    "    print(\"the test accuarcy is:\", acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:58:16.766415Z",
     "start_time": "2018-06-01T06:58:15.918700Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./model/model.ckpt-2900\n",
      "0.9856986\n",
      "1373\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAHiCAYAAADf3nSgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd8FEX/B/DPNwmEEggdpNcgRUFUrCAIiqioiPXBhhURRLE8+hMFK49YELAgioJgL9ixPKKPoKIgIL1IR3rvkDK/P3Yzs3PcXi65Szbl83698sp3b+bmdm9ub25ndmdFKQUiIiIqeAlBrwAREVFJxUaYiIgoIGyEiYiIAsJGmIiIKCBshImIiALCRpiIiCggbISJiIgCUuQaYREZKiLpIrJPRMpH+ZwVInJERCZFyKNEZL+IPBm/tS14IjJVRA6JyPSg1yVarNPIilqdsj4jK2r1CQAiMt6tn9VR5k9z6z9TRG72ydNJRLLcfOfFdYULkIgku9uQLiJP5Pb5gTXCInKViCx2d6oVItIhF09/XymVopTa7ymvnYj87L4Zm0VkYHaaUqoJgKeiKLeNUuohT5ljRWSp+0G5Icw23C0im0Rkt4i8ISLJnrSGIvKjiBwQkSUi0tXvRd1KfENE9rjlDfKk1RORGSKyQ0SeC3neNyJykvcxpdTZAPpGsa1x467/OBFZIyJ7RWSOiHTPZTFWnYrIfSKywC1vlYjc583MOs0/rM+jFeX69KzLT27jv8/9W5rLIoYrpRp6ystumPd5/hIBQCm1TCmVAmBaDmVucD8n37hlHiMin4vIBnF+dDX0Zo5UD256F7cuD7h128DvhSPVv1vOKhHZKCJXeh6vJCKzRaRC9mNKqcPutr6dw7aGFUgjLCLnAHgaQB8AFQB0BLAyhvKqAfgGwKsAqgJoCuC72NcUfwHoB2B2mNfsBuABAF0ANATQGMCjnizvApjjrs9DAD4Skeo+rzMUQDMADQB0BnC/mF+GDwKYAKARgEuyd2j3g7FSKTUr75sXN0kA1gE4C0AqgIcBfBC6A+WSALgOQGUA5wHoLyJXxbaaAFin0WB9Hm0oim59evV3G70UpVTzOJQ33FNeilIqM8bysuB8l/fySR8Kn3pw24FP4HxeqwCYBeD9CK8Vqf5fANADzmf1lewfFwCGAfiPUmpvXjYuLKVUgf8B+BXATXl87lAAk0IeewrAxNw+LyRdAWjqkzYdwA0hj70D4CnPchcAm9w4DcBhABU86dMA9PUp/x8A53qWHwfwnhtPAdDcjd8DcAWAinA+PJV8yrsBwPQg6tazDvMA9MprnYbJMwrAaNYp65P1mec6/AnAzXl87ngAT+T0WG5eE0AnAOt90pLc+m6Yi3q4FcCvnrTyAA4CODZM+RHrH86Pp+zHNwGoAaA9gG9y8x5F81fgR8LuL4qTAFQXkb9FZL2IvCgiZT15donImbko9lQAO0TkVxHZIiJfiEj9eK97iFZwfoVn+wtATRGp6qatVPavpb/cxy0iUhlA7TBlZeddAOAcEakE531bBOeD94JSalectiWuRKQmnA/5Qs9jua1Tb3kCoIO3vHzCOg2D9Vms6nOYiGwTkV9EpFP2gyJS363T3H5v9nO74f8UEb+j17iIoh6s+lbOUMgKhKlT5Fz/W0SkjYi0gXN0vhPO0fGdcdgUSxDd0TUBlAJwGZwdsS2AEwAMzs6glKqklMrNSQt1AVwPYCCA+gBWwelqyE8pAHZ7lrPjCmHSstMr4GgpIc8PzTsMzvv0PwAvwXnvjgfwhYi8I844eP+8bkS8iUgpOGMjE5RSS7Ifz0Odeg2F81l9M/Y1jIh1GoL1qcvxPj80b1Gpz3/D6ZKvA2AsnPVrAgBKqbVuna7NRXmj4HQN14DTBTxeRM6I8zp75VQPua3TSHn7AhgJ5326FsDtAH4AUEZEvnXHks/Ky0aESopHIbl00P0/Wim1EQBE5Hk4jfBDvs/KuczJSqmZbnmPAtgmIqlKqdA3GiIyBc5OAwC3KaXyMqC+D06XU7bseG+YtOz0cOMI+zzph0LzKqV2ALjSXe8EAD/D+YA8AOcX+A0AZovIVKXUojxsR9y46zcRwBEAcfnScb+8rgPQQSl1OEI+1mmcsT6tcrLTi2x9KqV+9yxOEJGrAZwPYHQey/OOw38tIm8DuBTAL+Hyi8g+z2LLPLxkxHpA7uvUN69Sai6c7nKIyDEAngNwGpwfWncB2ADgZxFpoNy+6Lwq8CNhpdROAOvh9PfHy7yQ8rJj8VmH7sqcSJCnM9rgdKW18Sy3AbBZKbXdTWvsPYPOTT+q+819PzaGKStcV92tAGYopRYAOA7ALKXUEQDzAbTO43bEhdvFOA5OT0cvpVR6HMq8Ee6JNUqp9ZHysk7ji/VprUuRr08fCj7fkflRnrJP4MrNEXf283OqB6u+xbk8rgnC11PU9Q9gBIDBSqmDMHW6Gk6Ph9+JfFEL6hKlNwEMEJEabj//XQC+jLG8niLS1u0+exjOSQ8xjceISGkRKQPng1VKRMq4v3QB4C0AN4lIS3cbBsMZmIdSahmAuQCGuM/pCad76mOfl3oLwGARqSwixwK4Jbssz7rUAHAHnK48wOly7ywiKXDGofJ8dnmcvAKgBYAe7oc1JiLSG84Jd+copeK2bazTqLE+bUW6PsW5tKabu61Jbn10BPBtDGVeJiIpIpIgIucCuAbA53FY1zIAsi8lS3aXs0Wqh8kAWotIL/c5jwCY5x1GyRZt/YtzJU8ZpVR2+7QKwNki0spdx+2xbm+BnZkXchZZKQAvA9gF58yzUe6GZqfvg9NdFe65QxHmDEo4ffb/wBlA/wJAvWie50k/6sxLOGf2qZC/Tp70QQA2A9gD54dAsietofv8gwCWAujqSesNYKFnORnAG245mwEMCrN+bwG43LNcD8Dv7vY+F5L3BhTgmZdwLhdQcLqI9nn+eue1TuF82NNDyhvDOmV9sj7zVKfVAcyE0926C8AMOD+IstPru3VS3+f543H02dHT4Iyj7oFzUtNVYZ73E3J5dnSY+lTR1gOArgCWuHX6EzxnVwMY4/3MRap/z2vNBdDA81gXAKvhHJFfFZL/qPcoqropqA9BHD9MgwHsdz9I5aN8zlL3A/ZGhDyH3A/U40FvY4zvz/fujvZD0OvCOi2Zdcr6LF716a7za279rIgyfzO3/g8g5NIxT56ObgO4C0C3oLcxhvcm2d2G/QCG5Pb54hZCREREBazIzR1NRERUXLARJiIiCggbYSIiooAU6GQd5yRczgHogHyf9WE8rwcEwPoMUn7UJ8A6DRL30eIl2vrkkTAREVFA2AgTEREFhI0wERFRQNgIExERBYSNMBERUUCCuJVhkZRYKdVaXvpiYx0v6zzOSjt18B06rvLmb/m7YkREVGTxSJiIiCggbISJiIgCwu7oCJJq1dTxosfrW2nzO43W8cKQ250fvthzG+M382XVKEar/nOatdyzmxk2eKrGbN/njd9TW8evPN/TSqv6GoceiCh3eCRMREQUEDbCREREAWEjTEREFBCOCUdwaFKyjpe1GGOlPbDpdB3PH3CclVZvzVYdZ+TTulF0vJeWLXniWB3Pu2SEle/mNefruOPd/ay05F2mFi8a8V8df/3Is1a+Dg3u1XHDwRwfLgiJVavoeMO/jrXS+vX7VMd9Kq6z0lr+7yYdNxuwVseZ23fEexWJIuKRMBERUUDYCBMREQWE3dEhEps20vEdDb71zffL8+11nPrrDCuNXdCFx+aJ5jKz5e1e0fEJzw+y8h3z3K86ToFdn15TWlXSccPlNa20yy+cruOZgxNzv7IUFXVGWx3XeX65jj+rO9rKl4UsT2xbcNZrOr7+82463nlGnFaSsOFeM2SXWcY8Xu+JX8Pkjs3BS8z3cdlP/wj7eGhaYcEjYSIiooCwESYiIgoIu6NDLL+1lo67l9ur4wc2nWzlS33bv8uSgrP1dnsmrG/amjOYm355l47Tnou9S+yhN66zlmf3H6nj0/reaaVVH8OzpfPqcHd737vqua913Cd1tY43Zx628vWcd6OOT6651kobUXuajv9Vw+zLLyEtpnUtyUK7fj8fMFzHXf53Z2j2sM9LvGOzjq+p+7uVL1HMoEKmso8fTy47SsczhzXU8allR1r5ZgxrBD9PTr1YxzV/Ed98Vaat13HGuvW++aLFI2EiIqKAsBEmIiIKCBthIiKigJT4MeHD59vjTUt6v6Tj7VmHdOy9JAkAUiNcxkLBSb10g7X84V4zi1LabTPj+lrV59m3z0qCuSyp4qUb7cz2hGuUg13XmrH9D598xkqrmZgcmh0A0OmDe63lJveaffT3W+1zBTBkGii+0svZx3S1k0w9Le7yqo57/nixlW9C4+fDPidUgueYMeuoi87Mvte8lJkdLQGlrVxNS9kzp3ldc8mL5nmXmNf6YF8NK9/Ye3vpuAzHhImIiIouNsJEREQBKZHd0QltW+r4lhGfWGlZUDo+7ynTvVX9bV5iUhSUHWh3Z715+oU6ror41mHyV3b39rg9dXUsokKzE4CkOrV1nLVzl5W2/9zWOv79aTO7Wboqa+Xrt76jjqd900bHTYb4X3Z2xq2zrGVv1+a69Ko5rTbF0eS0z0Ie8e+CDspNazvreNtF9vqV2RrfWbd4JExERBQQNsJEREQBYSNMREQUkBI5Jrzk9vI6viJli5XWcd4VOq71wVIdZ+b/alEcZC5aZi1XXVRwr/2f//bQcf3mmyPkLDm8Y8AAcNtPP+l40mb7sqFTK5i7UKUrs8etyjhk5VtzV1MdN/gtuulHs0KmOfRe4jL6Q1NvDRD/O/xQ3pzwykBr+aQLFuj4tfo/WGnnLDDf2+vWmzH+qr+VsvJVXbBfxysvLW+lNb7fe87Inlyvb17xSJiIiCggbISJiIgCUmK6o9Vp5lKG9899WccLj9gzr1R4PEXHmdtX5P+KUbHRts1KHe84VD5CzpIry/O7f2Kjb3zzee+I1PvR+6y0Kr9Fd6nZ9ptMd/foGs+GpJqZlJpMMLOsZURVMsVTn9Xn6njOBnOZX6NXl1v5tr1qLhW6pLZ9B7OUVf/oOG3Pqqhet3EhmfSQR8JEREQBYSNMREQUkBLTHb3zoYM6PrG0mex72PbjrHzy618Ftk5UtB3pdpK1PLmxGeY4f9EVodlLpIx/7BtqvHy9mfz+7utLh2bXWj69VcdVVkbX/ZzY1L5h+ydDzI0fQm/60OqnW3XcZOWcqMqn/LGzXy0d1/9rvo4jXpGydWuk1CKFR8JEREQBYSNMREQUEDbCREREASm2Y8IJFSpYyy+1etuzZMaEPx3d2coX7Z12JMm8dVmntLbSll9nZmmpsNTEdScssfJlbt8R1WtR4aSSxFpOFvOZWLPSvhF4GlYXxCoVet5zLtIiTE6Vl0uF1lx+jLV8TKK5+9LmzINWWvOHd8b0WhQ/ax4238cH95zkm887+1XV14rPXe14JExERBQQNsJEREQBKbbd0Rv72JcenVD6Jx3/ctj89oi2WyOpQT1reXlfM7PLwute9H+iuac8XrqpiZU0pVWlqF6bCqdN19s3FlibcUDHLUbutNJ4A5D84b0saW7/0Vaa9yYNFz57v5VWcyVv1BBvFd+xp6DqdetFOp7c7Evf5/112oSoyi91num2fn5gYyvtv1eYbuzQm7gUdjwSJiIiCggbYSIiooAU2+7offWzfNNumHajjpthdlTlLX6surW8vOtLOl6dYZ95ecmLpuvruEsW63hCw/9a+V55sr+OGz5UfM72yw+JNe2zjbdeYLr2d5vbyyKjzmErX6M623R8ee0/dTxifhcrX5XPyum40idzrbSsQ3a3c7aLm823lh9cb7rfMhcvD81O+WDpUP8hnTmeYada03ZbaSrf1oiyHR5cU8cfj6um454h93CPVrqn0u6ovNRKG3vleTpuMITd0URERBQFNsJEREQBYSNMREQUkGI7JnzjuT/6ppVal+yb5md8hzes5e1ZZhz46iH2TcdrjzeXP+yeaMZF0h7ra+X79hpzo/GB799kpWXNs2fXKom23m5uyv7G/S9YaceVLhWaPawey8w1Yu+tP1nHHRuusPKNeWaaju+/x56156exZj321TePf13jJStf089N/abhj6jWj3JPTmyl48lnvqLjtRn23FcPDLhbx8lzZub/ipElYbo5t+LNa81+uGv8VCvfZRXMGG6FBP87a0Xy1Q3DdXz1SvN9XHlC4T/XhkfCREREAWEjTEREFJBi2x0dD8vGtNfxSaXt7sWz/32Pjiu/7d/lkbFps46r1U610pokmQnmN55VxUqrOS9361pUeWc82jzC7mL+rd0oHX+5375E7PqR1+i47oerfcvP3Gje/+QsM2/VumR7SKJH/ct0vGSwfdnLhw+M1HHb0maXuX+T3W3d4kFz2QRnyIofdUZba/n+tybquLyYLuhLXrJnxar9FWfFKjT+MJfzTW5p78tj7rxYx0cq+hfx1+2jfdPqJpn9eV89c2OVyrlZx4DwSJiIiCggbISJiIgCwkaYiIgoIMV2TPj9cfa0hP++30wfmVE+uknr+p5hLnM67sfbrLTm35jT6qVObfuJmWZEcNXNZnrF+SfYd1tq/ev1Oq4/umSOX/39RAUdL2433kq7fMX5Oj50qT0Naa1t5v3Ky03Z1WF7esvM5St13Ox6O+89312h4x9afaLjftV+tvJd1M+MSdZ9qmTWZ7wkefap28Z/YKWdWcZMI9p1gDk3o/YnfM+DdPCS9tZy2U+ju0yv5qjo6q3Lgtt1/MoLI620tFLm0ibv2PGFT5wYVdlB4pEwERFRQNgIExERBaTYdkfX/nGHtbzsbtOF9dtlz+n4gnn3WvmqvGkuN8pU5jfKkrNft/KN+dncVPqs8vYdPbZmltdxxzJf63jGYbHy1b/cvgtPSZFYraqOnz3xIx0/sa21le9g9wM6ztq/P1/XSTyXLG2+2e7C+r7FMzpuNnmQjo9putXK97/bTb5Tqt1jpTUfY/JmLrNn66Kj/d2vgY67lbPvgNR7ZXcdV/x9nY7zMiwRL4e7m9nY0gdut9Iq3Zau44w161BcnfuYPTwzqV1nHTd+z7wnmYvydpej3Q1Mc1Uz0R6eyoL/XfMKOx4JExERBYSNMBERUUCKbXd06A0QLplkugeX3mgmfX9v6DNWvvv69NTxlamTPSllrXwDKq/RcbqyZ3pqUepI2HV6uO+t1nIpzAqbr7g7eKKZJatr2Sk6fn5VmpUvef/qfFsHCZkxa/VD7XS88Cb7LPb2s/vouPmDi3zL7P6uybfsypettGZVbzZxyNnXdLRTzl6o4x8PplhpB24zs8tl/pO/N3Df0u9081rn7rTShrU23w/HlZ6u45qJ9merZ/JV+bR2hcuGw/ZMc3/dZGa8e+nS5jr+oXtLK1/GuvVRlb+vkelyzuuNHgojHgkTEREFhI0wERFRQNgIExERBaTYjgmHajjYXHqUlmBmXll4vT3+936TbzxL9jiwV6byPyX+x4NldPzYv2/UccqPc6x80c3bVfyU/taMhbf+oa+O3zhzvJXvgd5mDD317Rl5eq3Eqmb8cFsPMy51Qr+5Vr6v6ryk49PmXmml1bh8rY6zDh2Cn2qXmUtRTrvyDiut+fum7ovuxRT5yzvj0pv1x+h44IbTrHx5ucQlsaJ9e55tl7bS8a5u5lK4hR3fCHnmn75lpiszM97DmzvqePoLp1j5Ki0r/DeWj4c5I+y7XeGZaTq8o7K5jHP3l/b36lt/2vXrZ1k3s48Wp32IR8JEREQBYSNMREQUkBLTHe3V+JGZOr5kbE8rbVXvujqu38VchrRsXU0rnzpo3rq639ozYVX4dZWOy2/+3Twnj+tbnNX9xLyP+8+wLzuY+rS5xOH+uzpYaQt2HqPjNStr+JZ/dltzSdEX9Ux31tPbW1j5Wr/aX8cNh8+20iJ1QfvlqzzB7oIsTt1n+WXLCeazkOXZW66rNt3Kd9UoU1f1p5h39kjFRCtfxnVmlqb2NdZaaZNrm89WgudYJHTmpee2m1nc3n/TvilM2a0mb+okM1xSCSWj+zlUxZUHrOXP9lfTcc/yZgbDwdXmWfkGd7OX/ZQSU7/pIV+mi9PNUFD/gXfquCyiu4lEkHgkTEREFBA2wkRERAFhI0xERBSQEjkmrDLM/VYyVttjRfWeNMvqSfN4M/wTdfmZOWchV9nPzJjNy390stIeP7uhjnf0sMeblnV8S8dj69WGn+UHzVh+i7fMZUPNXrXrs/5qc2Nxjt8GI/Xv8GdNtAmZoXBxL89lhb1MmBByTBHtnXWe2nacjsfPPN1Ka/HvlTqutT26m8+XWDPssd0Hv75ax1nd39dxz5QteSreOw4cWrf9l5rX8n6nFAU8EiYiIgoIG2EiIqKAlMjuaCqcMjZuspZT397kie283RAyO48v04fVyHPpSJA3gKfwKk009dM+dYBJOGeHle/c+uYOaY/VMJcbrs04aOXr+s3dvq/V/HVzOVnCSjM0kbbdvrMZh5byruld5rKtwRlmFrpd539h5busgpkBLdLdkbyXIf16oImVJi9W9yytQlHCI2EiIqKAsBEmIiIKCLujiajQqfGi50xk+x4r8N564yKc7FtGGmb6pnnPw2aXc/5rcq/pmp58b3UrrdRiUwPNks0QVN83+ln5qs03g0ihZ0CXKQIzY/nhkTAREVFA2AgTEREFhI0wERFRQDgmTEREgfmgRS3PkonroWTMUMYjYSIiooCwESYiIgoIG2EiIqKAsBEmIiIKCBthIiKigLARJiIiCggbYSIiooCwESYiIgoIG2EiIqKAiFIq51xEREQUdzwSJiIiCggbYSIiooCwESYiIgoIG2EiIqKAFPlGWESGiki6iOwTkfJRPmeFiBwRkUkR8igR2S8iT8ZvbfNHNNtTVIjIeHdbVkeZP82t+0wRudknTycRyXLznRfXFY4zEbnJXU8lIk2DXp+8KOl1GImIJLvbkC4iTwS9PnlR0r9z412HhaYRFpGrRGSxWwkrRKRDLp7+vlIqRSm131NeOxH52X2zNovIwOw0pVQTAE9FUW4bpdRDnjJ7iMgCt8xfRaSlJy1ZREaIyAYR2SkiL4tIqQjbe7aIzBaRPSKyUkRu9aS1EZGFIrJNRO72PF5KRH4XkXresnKxPQVCRH4SkUPu+7RPRJbmsojhSqmGnvKyv9T3ef4SAUAptUwplQJgWg5lbnA/I9+4ZYqIPCQia906eE9EKnpes46IfCYiO0RkvYj0jbC93gYi++96T/oL7mfiNxGp43m8t4iM9JallBrnbk+gRKSKiEx298c1IvKvXBYRWofJIvKG+15vEpFB2Wkx1OExIvK5u88pEWnozRzpNd30LiKyREQOiMiPItLA74VFpKGb54D7nK4h5awSkY0icqXn8UruPl7Bs62H3W19O4dtzVci0kJEporIbhH5W0R65rII6zvX3dYJIrLF/RvqzVxIvnMjlRVYHRaKRlhEzgHwNIA+ACoA6AhgZQzlVQPwDYBXAVQF0BTAdzGuYzM4b3pfAJUAfAHgcxFJcrM8AOAkAK0BpAFoB2CwT1mlAEx21y8VwJUAnheRNm6WYQDuBdAGwGARyb7T9SAAHyul1sWyLQWkv7uTpiilmsehvOGe8lKUUpkxlncdgGsBnAGgNoCyAEZ70icBWAWgJoALADwlIp0jlLchZP0mAICItAdwIpy7lU8H8KD7eCqcOn4kxu3ILy8BOAJn+3sDeEVEWsVQ3lAAzQA0ANAZwP0S+xFtFpz9vFduX9P9jvgEwMMAqgCYBeD9CK/1LoA5cL5PHgLwkYhUd9NeANADwHlw3qdE9/FhAP6jlNqbl43LL+531mcAvoSz7bcCmCQiaTEUOwJAOQANAbQHcK2I9IlxPeP5nZtTWYHVYaFohAE8CuAxpdQMpVSWUuofpdQ/MZQ3CMC3Sqm33V8te5VSi2Ncx24ApimlpiulMuD8aKgD4Cw3vQeAUUqpHUqprQBGAbjRp6wqACoCmKgcMwEsBpD9y6wRgKnue7AcQH0RqQ/ny2ZEjNtBjh4Aximl1iml9sGpzytFpJyIpADoBOBJpVS6UuovAB/Bvz4jaQRgulLqMIAfADR2H38SwDNKqd2xbki8idPF2AvAw0qpfUqp6QA+h/OjJa+uA/C4Umqnuy++BuCGWNZTKbVZKfUygJl5eM1LASxUSn2olDoEp8FuIyLHhhbiNk7tAAxRSh1USn0MYD5M419eKbXA/ZwcAVDV/fHVSCn1QSzbmE+OhfPDc4RSKlMpNRXAL4itfnvA+aF8QCm1GsA45G1/8Yrnd25OZQVWh4E3wu4vjpMAVHe7RdaLyIsiUtaTZ5eInJmLYk8FsMPtctgiIl+4jVhMq+r+hS63jpBe1z3isSilNsP5Zd1HRBJF5DQ4v9anu1kWADhXROrC+WW5As4H7H6lVHqM21FQhonTnf6LiHTKflBE6rv1mdv66CdO1/CfIuJ35JMb4eorGc6Rk3ge86a3hr8a4gx7rHK7yLLHyhYC6OB+nrsAWCgiJwForpR6Jw7bkR/SAGQqpZZ5HvsLQCsg93UoIpXhfOn/Fa68/BDFa7byprndqit81qkVgJUhR0PesraIM4TUBs7R+U44R1Z3xmFT8oP4PKY/33n4zg0tN6f9Jdry4vKdG0VZgdVh4I0wnO6uUgAuA9ABQFsAJ8DTraCUquT+Go9WXQDXAxgIoD6cbsV3Y1zP7wGcJc74X2kA/wegNJwuGACYAmCgiFR3u4+zK6/c0UUB7vo8AuAwnLGwhzzdzPcCuB3O0cfdcLpM9wJYKc445f9E5PIYtyc//RvOEV8dAGMBfCEiTQBAKbXWrc+1uShvFJzGsQac7sPxInJGjOs4BcDN4oz1pbrrDADl3C/bXwA8LCJlRKQdnKMev7pcAudzewyAs+F0Pz8PAEqpBQA+BjADzmfxaQAjAdwpIneKc97C2yJSKcbtiacUAKFH6LvhDBXlpQ6zx7i9Zery8klOrxlxG8OUFSlvXzh1OhbO0eTtcHo9yojIt+KMJZ+FwmMJgC0A7hPnPJNz4RwR6s93Hr5zvwHwgIhu1MhWAAAgAElEQVRUEOeEwhvhv79EK57fuTmVFVgdFoZG+KD7f7RSaqNSahucL7DzYyxzslJqptvV9CiA031+IUFEpog5oaZ3uDxKqSVwGvYXAWwEUA3AIgDr3SxPwhkzmgvgVwCfAkiH82EPfb1j4Yw/XQfng9AKznjVBe5rrVFKna+Uagdn7OYxOA3zs+7zLoIzhlwld29LwVBK/e4OARx2x0Z/QQz1qZSarZTarpTKUEp9DWds51K//GKfIOV3tPYGnB9CP8E5Wv3RfTy7PnvD6UpeB+AV9zXXIwyl1Cal1CJ3KGUVgPvh/KjMTh+hlGqjlLoSzvj/NDj73q1wjo4XwxnfKiz2wRku8aoI54dgXsvLLiOq8qKsw1heMzfbGDGvUmquUqqTUuoUON8JN8I5Cel1ON89fQBMFJFwR6AFzu1NuwTOuQ6bANwD4AP4fL6jdCec793lcL6z3o1UXkF/5+ZUVpB1GHgjrJTaCeeNiOck1vNCysuOw76BSqnunhNqfM94U0p9pJRqrZSqCmAInC7kmW7aQaVUf6VUHaVUYwDbAfzpcwJRawBLlVLful/cSwF8BaB7mLyPAHjd7cI+DsAsdxxxPZwTzooCBZ/3Pj/KCzlBKuzRmvu+D1FKNVRK1YXTEP/j/mX/ELpQKVXd3TGrAvgjlvUTkZoAboPzo6o1gHnuF+JMAMdHWXZBWAYgyT2ZJVsbOO9Rrrn7+Ea3jKjKi6YOY3zNhd40d/igic86LQTQWDxnyEZY/xEABiulDsLsr6vh9PZVD5M/EEqpeUqps5RSVZVS3eD0XEX7+Q5X3g6lVG+lVC2lVCs4bYtveQF850YsK0SB1mHgjbDrTQADRKSGO5ZzF5wz92Ipr6eItBXnTOSH4ZwcsyuWlRSRE90x3Opwzmz+wv2FlX1JS21xnOq+5hCfouYAaCbOZUridtVeCHv8CuKcQt8JzpEY4HSrn+1+mTcDkOsvp/wmzin93dxu3CT3V25HAN/GUOZlIpIiIglu19k1cLrqY1nPKiLSxH3/W8LpfXlMKZXlprdwu9ZKi8g1AM5184Qrq5M7TiriXD72HzhHA6Geh3NyzwE4dXmymJPA8nw1QLy546OfAHhMRMq7Xf8XA5gYQ7FvwTnTv7LbE3QLgPGxrquIlIEzlg8Aye5yNK85GUBrEenlPucROD+KloS+hjs2PhfAEPdz3RPOj6aPQ9blHABllFLZ313Z+2srdx23x7q98SIix7vbUk5E7oUzlDI+hvKaiEhV9/uxO5xenpivoY3jd27Esjx5Cr4OlVKB/8H5hfEygF1wukdGuW9Edvo+AB18njsUwKQwj98O56hmJ5zT0etF8zxPugLQNOSx6XC6oHbAqcTynrSOAFYDOABgKYDeIc+dAuD/PMtXwDkBay+co9qnASSEPOdHAKd4ltvA6SrZBmBQbranAOuyOpxfl3vd+pwB4BxPen23Puv7PH88gCdCHpsGZwxuD5wfKleFed5PAG72KbMTgPUhj6W59XQAwJow7+ddALYC2O/W+0kh6fozCeds/H/cstbBudSpQkj+zgC+CnnsBffzOQNA3Zw+fwVcj1XgdO/th/Nj718x1mEynCGAPQA2h77fealDz/tk/UX7mgC6whkfPei+dkNP2hgAYzzLDd08B93PTdcw2zcXQAPPY13gfCdsDP3MhnuPCrh+n3E/e/vgfDeFftfl6jsXzvfZBncfmAugWzTPy+kzj/h+5/qWFWQdBvIBiPOHaTCcL4pdoW9qhOcsdT9kb0TIcwjOF//jQW9jPLanqPzBuYxkH4AVUeZv5tb9AQA3+OTp6H557gr35VCY/uCMPe1yP3+Ng14f1mHc35tkdxv2w+kVCXyd8rANJfo7N951yPsJExERBaSwjAkTERGVOGyEiYiIApKUc5b4OSfhcvZ9B+T7rA/jfn0b6zM4+VGfAOs0SNxHi5do65NHwkRERAFhI0xERBQQNsJEREQBYSNMREQUEDbCREREAWEjTEREFBA2wkRERAFhI0xERBQQNsJEREQBKdAZs4qTq5ds0HHvCht986V9d6uJ+/yZr+tERERFC4+EiYiIAsJGmIiIKCBshImIiALCMeFITj1eh1eP/8ZKurD8Kh1nobR/GSpfbnZDcZbQtqWO13avZKUlHTBx8m5zU5q9F+yz8j3a5nMdX5Gy20rLVFk6TvvxJh03vWZO3laYiIoFHgkTEREFhI0wERFRQEp8d3RipVRrefHwNB0PO+sjHfdK2Wbl83ZB7846YqWd+ukgHadNPAAqHCTJfNyXP3OSlfZlz+d1nFaqjG8Z3x8sq+NdmeWstAe/ulrHD++zhyESD5vlenPSo1xjCierwwk63tXM1NXhSvZ7XmWpeZ/XdUm00mq02Krj6cd/qONndzS38k15oJOOy89eZ6VlbNyUi7UmCo9HwkRERAFhI0xERBSQEtkdffCS9jpO6LfFSlvS6mWfZ/n/Xvlkb5q13GzA73leN8o/3u7o8o3ss5evmX+DWfiiqpVWdoc5sznli7k6VocPW/maYkYc1pJC/T3xBGv5sfbmLPQ6STt13KFMhpUvCwrRyPLE91ZZaqUNGrtExwM3nGGlrelRU8cZmzZH9VoUWWLzpjpefG9lK63HCWbfG1V7ppX21QEzLPHIM310XO3V3+K9inHHI2EiIqKAsBEmIiIKCBthIiKigJSYMWHvOPD3L70Uc3nHTbhTx43+r/CPOxCQdeiQjquWty8d2/xzHR3XG/urbxnRjTJSrNYOPV3HY09/1UrrWOZIaHZX/s5ON7L2L9Zy8/+YO6Q1u4Fjwnm17bbTdPznkFd883nHfU+de5mV1r7GGh2Xuthcfgb7o1Mo8UiYiIgoIGyEiYiIAlJsu6MTW9kz37z4wihvalRljNp5rI6n9m5vpTVeMlvH7KIsGhKbNtLxG2kTrbSLvr+/oFenxEusWcNaXn5PEx3P+ZeZwSxZSuWp/C4LTJfl5lm1onpOej37srOlXV7zzTv01M90/G6FFjrO2rs32lUskdY8dpq1vORm0wXt7XJ+ZsC1Vr7kKeaypFT8baV98drJOn6xs9m3R+FYFHY8EiYiIgoIG2EiIqKAsBEmIiIKSLEdE148sKK13LxUdOPAT21rq+OZV7fScdaixfFZMQpM6XHmsqT/bD7HSqszYpaOOcZfMPZMSLGWlxxnLh3MQu7HgZtNvt1abv7gIh033LsqqjIS2ra0lpd1NJdDpZUqbaVdXcFclvRGh546Tv7anlKR7HFg7xgwANy5wYzn/t27gY6Tl/q/j97pLQF7HPjxZRfqOHTsuDDikTAREVFA2AgTEREFpHh1R596vA6HnfVRVE/xXoYE2F3QmYuWxWe9KDDeS9WG1n9Lx7c8dpeVr0o6Zz0rCCuHe7olj7NnrksUzzGBykI0zpx3uY6b9bfvXhZdCbasuYus5WkHTLfnsanr8lBiyXW4u+lmfu5fb+rYexkSYHdBZy6Nrvt4Ze/q1vIF5cxseEj7UofPdPe/zKmw4JEwERFRQNgIExERBaRId0dndbBv9v34hNd1fGJyaO7wvzfuqmx3Od/1ffgu6FJin13d5P2+Ok7e4f9bptp8c6Pxsp/+4ZuP8of3LPkWpU09lTrIc6ALQkK5ctbye5eP1HFW6Mx1ni7orAjnqL+0y8ysVemKLZ7nxN/YURfp+KaHX8yHVyi+Dg3YqWNvd/H5ne2bL0TbBe290YO3exsAGn11i45XXWBmOet/qV1G2pSoXqpA8UiYiIgoIGyEiYiIAsJGmIiIKCBFekz4n05lreUTkr1jSvGVHjJEteiK0VE97+90MyZ8R7+rrbSsl81dZDhenD9+PW+Ejh/dYm4UX+G9GUGsTomz9OUW1vLxpaf55m36nRnXe7HD2zo+t+x+K99bo7vruPre/L207Jgv15qFh/P1pYo87yVJADCjrRmbPXWuGQdOjXIMOLS8P4eYmbY63XSLlValvqcpu8Dz+KzC38TxSJiIiCggbISJiIgCUviP1SM4XDU/LkqIL++k79+2+tBKWz/S3EB8wMArrLSsQZXNwiLTfaMO2zcdJ0CSzfVoy8fZE/BXS/xTxzPvOlHHCZiT/ytWQiXVqa3jP7qMCkktAz9Jm82+0v8HM9NRpVp7rXw1xhbc0M2OjvUK7LWKujWX+qftmmVmuIr2pgreS5wA+zKktJCZrzZ5bhBR1PBImIiIKCBshImIiAJSpLujl1xuTwBf+DunbXWTTDfq5LTPrLSEL83vo1Zv9ddxowd5o4FQiXWO0fGyzuOstMe2mZt6lF5hZldS1e0J4L3UoUPWctbevT45KZz9beroODXBv/s57cu+9vID5rO96S5zJnu1d+yrIJCVGeMaRm97aymw1yrqatbZ6ZvW4JHcf2+lf2bvo2mv+pdx7vmzwj5eLcJzCgseCRMREQWEjTAREVFA2AgTEREFpEiPCYfe2Sh0Vis/N63trONx9X+M6jnPbLcvffnlfHOzb4g9brTo0VqeNLNSX3a278LivXwplHfbFl5nnte8Qj8rX+iNzEui5bcc45uW4Lkbz4hfPtBxk6Sy4bIDALou6mktl7utqo4zVq7OwxqWLGsuMXEC/MdUKy4p5ZtW64Vf47lKeTboks91HLotPx0y619+0WYdZ4DiIdJ47pqQS5K+rf2KT87Cj0fCREREAWEjTEREFJAi3R2druxLFbKivEhp20Xm0qAzu/e3Ez1d2lXm7dJxwvY9VraM9f/4lp924/qwj9/X4jpreen/peh40dljrTRv17p3u/64+HkrX9dl9+m45qjC0YVXEJIamJmMxl7xqm++wdUW6LjpF3fruPoMeyjjUBXT1TjiDru8kW+eYxZ6m8tvIn0GSprEtCY6ntTVvH9Z8B8jqrw0PV/XKS+82wEAx5WZrePQbXli5YU6Lr16Tf6uWBGw+Z/K9gNtTZjY3AzfZUZ5AwfvcwBg8b2m/FUX2N3Pd24wN3sYVdueTauw45EwERFRQNgIExERBYSNMBERUUCK9JjwH4ftSwZOSvbJGOoDc2lBxYcPWkkJ0+fq2DvCHI8pMTMXL7eWmw9I1fFxr9xspS3s+EbYMiok2Jc1HakYhxUrgjZ2r6vjlqXNtJJjdh9r5fto0Hk6PvaXxTqONBXlsz/Yd7T68mtzg/kWz/TRcaOrOSac7UCzKjpun+w/Dtx+Vm8d156zzkorDJf2bOxa01qOtC0VSps7mvHeZkCVWSHNyQUmvOOrL3V8zzt9rGzlPbtRqYu36nhG2498X+vER2/3X5EhHBMmIiKiKLARJiIiCkiR7o6+5U378qI5fUdG9TzvHYsmj6thpb15rbnsAH/Mz/vKRSFz124dV5xazk7smK8vXeSV3mu6CW/4+3IdH37cnj2r9FTTNRXtkIJs2Oabdstx03X8X1SIskTKVrufudQvY+OmANfESKxpvgO+ffCZkFRzF6hVGfbdtXY/W9+Tq3BsS5BCZ7g69eLLdOztWr7g5uhmtzp17mXWcpnR5hKlalPs12o+ywwxfnXA/85dhRGPhImIiALCRpiIiCggRbo7ut7j9gxRx7e+SccLznwzqjJ6pdhdj70mjw+bL/RmEc/vaKzj198/LzS7pjwncEvIiZa3XvW1ju+oZN/cwft60d6YoriRZHO6e2Idu5s59e0ZOs40Jy8jCRtift09HRv7pr35fjcd10PJmaEsXvYfb2YcS/4n9rqKh8VDGuq4aoJ9Yw/vLFndptxtpaV98Ue+rldRl3q+mRnr/Oama3lzp+pWvgprzXnxyVPM8FEq/GfWCp1Na1Rt093t7caOVEZhwSNhIiKigLARJiIiCggbYSIiooAU6THhUE1uWa3jtOH2jCrDOn+o454pW3Jddui47B2Vl5q471L4SfD8zol0l6fQFL+7KF20xL7hfMNXzSxc9j2lir6EhuZOSQ9Ped9KG9ridB2rw7HPV5RUz8zA1elhe6w3w/PONnp9pedxypb8lRnLG7fbXLpzS6o9K9baq817eezMqlZa5rbt+bR2wLqHT7eWf7rVXIpUNcHcKWl1xgEr39gdZ+o4rS/HgPPKe+ekalHeRamk4JEwERFRQNgIExERBaRYdUdn7jGz8YR2Hb3Z3syEtWv8VB3XK213gXUt6z+xf0F6a4+5lOPFly7Vce1P7ZuHZ2zdipLg5GT7Zh3pX5mJ9pMv3aVj72cgkqQG9azljl8u0fEdle2Z0toPH6TjWht5WVJOtmWYmcS8l/gAwNIur+n4+DsGWGmNx3k+28o8LyPkUiYpZW5ikljTXO6y5Zz68HPzld9Yy5UTzKxKw7a31PG7755t5WvwuvemKyVjXysKQi9zKsp4JExERBQQNsJEREQBYSNMREQUkGI1JhyR545Ik1ua8YTElmdY2e67tFpUxb12o5lm8qTkvF0c1OKH23RcZpl95496T5ixxxqe6RFL0mUxWSvX6rjZZPuSs6U9X9bxL3PMHVRum3WNlS9jo7k71THHmkvTRjZ/z8rXICldx8d/NshKazaS48C58dNAcznQvyct9M0379bR1nLCrWbcf2fWQR1ftri3lW/dFnM3nSWdXzfPh33eQOh4tJ9PXums47qv2HVd3C77o8KHR8JEREQBYSNMREQUkJLTHe0jc9Eya7leyLKfx55oF/NrN8PsnDOVYCr9iI6b9f/dSjttbn8dTxz8nI4XnTk+qrI/3W8PO9z0vOnGbjaa3c+xKDXLXNZz+d/nW2mTmnym42QpBT+pnkuIvm/1sZ3YKvfr1H6W3aWdOsZcRlVjqtkPS+gNyyhAPBImIiIKCBthIiKigJT47mgqmqq+/puO73r99Ag5o1MT7IKOl6y9Zta5g2fZM9C1f8icef7yjWOstA5lojv333uDCK9XXrvYWq681JzxXnuOfSOJjI1mhjR2QRc9NX8Kmb1sSDDrEQ88EiYiIgoIG2EiIqKAsBEmIiIKCMeEiajA1HvSjL0Pe/J4K21YjGXXijCuX5JmmisJMpf+bS2fOvcyHe+aZWZETIWdrzDikTAREVFA2AgTEREFhN3RRERUpKWeb7qdi0IXtBePhImIiALCRpiIiCggbISJiIgCwkaYiIgoIGyEiYiIAsJGmIiIKCCiFO8hQkREFAQeCRMREQWEjTAREVFA2AgTEREFpMg1wiIyVETSRWSfiJSP8jkrROSIiEyKkEeJyH4ReTJ+a1vwRGSqiBwSkelBr0u0RGS8Wz+ro8yf5tZ/pojc7JOnk4hkufnOi+sKFyARSXa3IV1Engh6faLBfTSyIrqPsk4jiKVOC7wRdr9UxonIGhHZKyJzRKR7Lot5XymVopTa75Z5n4gscMtbJSL3eTMrpZoAeCqKctsopR7yrOtYEVnqfpnfEGZb7haRTSKyW0TeEJFkT1pDEflRRA6IyBIR6er3ou578oaI7HHLG+RJqyciM0Rkh4g8F/K8b0TkpJBtPRtA3yi2NV+ISDP3w+i74/kYrpRqGKa8KiKy1fvhVkotU0qlAJiWQ5kb3M/JN25Zx4jI5yKywd35rdeLVA9uehe3Lg+4ddvA74Uj1b9bzioR2SgiV3oeryQis0WkgmdbD7vb+nYO2xpXIjLJXb89IrLM78dOBKH76F0istItb4OIjBARPXc999H8xzo9qpxCUadBHAknAVgH4CwAqQAeBvBB6BdiLgmA6wBUBnAegP4iclVsqwkA+AtAPwCzj3pBkW4AHgDQBUBDAI0BPOrJ8i6AOQCqAngIwEciUh3hDQXQDEADAJ0B3C/m6O1BABMANAJwSXblu1/eK5VSs/K+efniJQAz41je0wAWx6msLADfAOjlkz4UPvUgItUAfALn81oFwCwA70d4rUj1/wKAHnA+q6+ISKL7+DAA/1FK7c3LxsXZMAANlVIVAVwE4AkROTGG8r4A0M4trzWANgDujH01uY/mAuvUNhSFoE4LvBFWSu1XSg1VSq1WSmUppb4EsApAnj8MSqnhSqnZSqkMpdRSAJ8BOCMO6/qSUuoHAIfCJF8PYJxSaqFSaieAxwHcADjdpQDaARiilDqolPoYwHz4f/lfB+BxpdROpdRiAK9llwXnQzBVKbUbTuPWWEQqwvkg/l+s2xhP7g+fXQB+iFN5p8HZud+MR3lKqc1KqZfh/yMhUj1cCmChUupDpdQhODtwGxE5Nsx651T/5ZVSC5RSfwE4AqCqiLQH0Egp9UE8tjVW7uf6cPai+9ckhvJWKKV2uYsC5wdR09jWkvtobrBOj1Io6jTwMWERqQkgDcBCz2O7ROTMPJYnADp4y8snreD8Ysv2F4CaIlLVTVsZckTzl/u4RUQqA6gdpqzsvAsAnCMilQCcBGARnA/eC54dIHDuB/QxAPeESavv1mn9XJSXCOeouj+cL4t8FUU9WPXtdsmtQJg6Rc71v0VE2ohIGzhfXDvhHB3H4ygibkTkZRE5AGAJgI0Avvak5XofFZF/icgeANvgHDW9Gs/1DYP7aAjWqaMw1WmgjbCIlIIz1jVBKbUk+3GlVCWlVF5PWhgKZ7vicvQUQQqA3Z7l7LhCmLTs9Ao4WkrI80PzDoPzo+J/cBqlUgCOB/CFiLwjIj+LSP+8bkQcPQ7nF+q60ASl1Fq3Ttfmorw7AfyulPozbmsYWU71kNs6jZS3L4CRAMYCuBbA7XB6D8qIyLfumNZZedmIeFJK9YOzzh3gdMUf9qTleh9VSr3jdl2mARgDYHMcVzcc7qMhWKdWOd7nh+YtsDoNrBEWkQQAE+F0x8VlY9w35ToAF3i6XcLlmyLOWX77RKR3Hl9uH4CKnuXseG+YtOz0cGN9+0Keb+VVSu1QSl2plGoD54t7NIABcLpFFgDoCqCviLTM43bETETauusxIk7l1YbTCD+UU17Pc/Z5/qI+4vaIWA/IfZ365lVKzVVKdVJKnQLnF/aNcE5geR3O+FYfABPdXp1AKaUy3S/munB+LMSjzOVweqpe9svDfTT/sE51Od7nW3kLsk4DaYTdL5dxAGoC6KWUSo9DmTfCHbBXSq2PlFcp1d09yy9FKZXXs04Xwul+ydYGwGal1HY3rbF4znJ104/qInfHNTaGKStcd/qtAGYopRYAOA7ALKXUETjjHq3zuB3x0AnOSRJrRWQTgHsB9BKRo06siFJ7AMcAWOSWNxJAe/cMxsRwT/DUZ0ouj7izn59TPVj1Lc5lGk0Qvp6irn84P1wGK6UOwtTpaji/vP1OKAlCEmIYP8xtedxHCwTrtBDUaVBHwq8AaAGgh/vlExP3V9VTAM5RSq2MtTxPuaVFpAyckw5KiUgZ9wgeAN4CcJOItHTHFwYDGA84l9AAmAtgiPucnnC6Mj72eam3AAwWkcruiT63ZJflWZcaAO6A090OOCezdRaRFDhjFnHb7jwYC2fna+v+jQHwFYBueSxvCpxGPbu8R+Cc8dhWKZUZy4q69Zl9SUOyu5wtUj1MBtBaRHq5z3kEwDzvMEq2aOtfRM4BUEY5JycCTp2eLSKt3HXcHsu25pWI1BCRq0QkRUQSxTkj9WoAU2Mo82b3Mwz36OFBxOEEPu6j0WGdhlU46lQpVaB/cE4HV3DOfNvn+evtybMPQAef5w8FMCnksVUA0kPKG5PT80LSFYCmIY/9BHMWYfZfJ0/6IDhjIHvgjEEne9Iaus8/CGApgK6etN5wzrTNXk4G8IZbzmYAg8Ks31sALvcs1wPwO5yTep4LyXsDgOkFXbd+7zWA+m6d1PfJPx7AExHKC7s97vt7s89zOgFY71PP1l+09QCnC2qJW6c/wbncIzttjPczF6n+Pa81F0ADz2NdAKyG8wv9qty8R3Guv+pwxsJ2ue/FfAC3hOTJ7T76pvue7ne38Rk4P0C4j7JOS3SdFsgHIM4fpsFupe+Cc6lHNM9Z6n7A3oiQ5xCcgfnHg97GGN+f7+GMa/wQ9LrkYp1fc+tnRZT5m7n1fwDADT55Oro74i4A3YLexhjem2R3G/bDufQi8HWKYp25j0be1qK4j7JO86lOeStDIiKigAR+nTAREVFJxUaYiIgoIGyEiYiIApKUc5b4OSfhcg5AB+T7rA/jPvED6zM4+VGfAOs0SNxHi5do65NHwkRERAFhI0xERBQQNsJEREQBYSNMREQUEDbCREREAWEjTEREFBA2wkRERAFhI0xERBSQAp2sg4goGlkdTtDxisuSrbQGLTfq+L8tJ+v42tVdrHw31Jyu48fvu9FKKzf597isJ1GseCRMREQUEDbCREREAWEjTEREFJASMya896pTdVx/wDIdv93wv3kqL1HM75fOCy+20v6ZWVvHTR6bo+OsQ4fy9FpUeCTVqqnj86custImrDKfsSqDS1tpas7C/F2xYiC964k6fm3cSB3XTyrr+5wsTzwhZF9OgJk//5TRL1hpVy3vY8pYsCS3q0oUNzwSJiIiCggbYSIiooAU2+7oxFbNreWPhz+r42qJpnsrC3mTpTJ1/F3LT+zEliacd7XJ969JA61sVRabW31WfGdGHteECtKRZmaooW/qFCutb9s1Om5xfx8rrdHV+bteRVFixYrW8snP/qHjnw401fFTUy6x8qWsMccO+xpkhX0cAGbf/6KOy4k9PIAkHn9Q4cBPIhERUUDYCBMREQWkWHVHe2fZ6f3aF1aatwu6IB1fOlHHC2580UrbmHlQx90b3G+l1R32a/6uWBGTVKe2tZy1c5eJDxwosPVYcYvknAlA3XHFatfKHwn2e1kh0Vw9MO5R0wXd9L08DtXcn3MWKry23HG6tTznoZd1fN2ajlbatIVpOk67eVb+rlic8UiYiIgoIGyEiYiIAsJGmIiIKCDFauAqo7zZnKsrbLbSDqt0Hf/fpg46nvNkOytfQoZCNA5VMmO9pwy0xyC6pprZkbqX2+tbxjGeceqHb3jXSps46QwdZ6xbH9U6FTdb+56m4z4DvrbSlh6opV77VicAACAASURBVOMVZ1fQcdZe//c7r6SUubylSZ2tvvnmHzGfsdI77dnRovtUlSyZu3Zby1OPK6/jCojvJXt/HraXE3bt03FeL1MkW0KZMtbyhr7mu3V/XfMu33juj1a+eXvr6Pj3ZY10fF27/1n5/rWqs+9rzz/PnG9z2cm36FjNnJ/TageOR8JEREQBYSNMREQUkGLVHV3ur3U6bjdygJW2v56ZuarZAHND73LI2829vR0vSyfaactO7qXjQXeZt3hxp9d9y+uVss1aHnZZfR3XGlEyuqNDL0PydkH3q7TKzuxZ7pHo300VD0c6H6/j748d65tv+IbzdKz+5A0bCoK3C3TnJ3WttESZq+P+T91upVVd/Vv+rlgJJGWSreWu15ohheG1zJDdioyDVr7XZpjLjV7v8oaOD6lSVr6f/20uWdrR3E574BpzmeI/nc3wVO2ZUa16oHgkTEREFBA2wkRERAFhI0xERBSQYjUmnLFxk45rD98UIWf+8p4WX2qZZ+q1TgW/LoVdYs0aOq718R4r7ahx4ICsvjAx50xUYBLKm0uZ1vVvo+M5bUZb+RYeOaLjGp8us9IyQfEWesnZkovMOR7Hj7pWx1XfKm/lq1nBHAvenGTuPtZyiP0dXnqdGeBNHtDMShtZ24zxN6tzAooSHgkTEREFhI0wERFRQIpVd3RhceDSU3Q87eZnPClljs5cwqljqul4bL1JUT+v3fD+Oq61O38vN/nlkuc8S+V8882Y1VzHzeI841NJllTPvvRo8f1mhqWll5ou6FUZ9ixl93W9RseZ2wrH0EZJkvHPBh3X7bXBN5/3/napnq+AjJB8WWeZbubrG/3XSpu418yglzbezJpXFGaq45EwERFRQNgIExERBYTd0XFw8JL21nL5/maGq9SE6Lqgx+xqbC0fM/oPHReFLpW8WndepajyDdnaxlquM2mJjjNV/r5DFRKi201qcRKmuElsZbr2b/n0KyvtgnK7Q7MDAF7bfqb9wK49YfNR0ZDVwT7LeeNAc7b7zakrrbSz/m1mSEydU7SGgngkTEREFBA2wkRERAFhI0xERBQQjgnn0e7ep+r43aeetdLqJpUNzR7W3RvMbFqrLqthpamMdaHZi6UjbfZHle+rNa2s5Rrbl/jkpOJg8YBUHfuNAYd6quYsa/lPc1oFbn3Rvqta3Qmecwq278jDGlJ+W3mbvSyHzJ2T2r460EqrP+nXglilfMEjYSIiooCwESYiIgoIu6Mj8N5kfvtr9kxJY1u+oONou5/7rjvLWl423HSxllvze15WscQ49GeVAnutxGb25WIJMP2aiWJ+t763t7KVr/Jv/+g4dLYfyp0qs81NM05cFl1X8v5ep1j5Hnna3CB+zj0vWmlNW92q4/qfNtVxmS/+AAXn7+fNMN+Vrewu5g8Xt9Nx/ceKbvdzKB4JExERBYSNMBERUUDYHR1i8wBzxvKFN03T8ZDqc0NyloKfXzxn8d306/U6Tuv3t5Wv3F52Qdf8wDOjWEf/fGVOLLgzWFdeW9NaThazm2SqLN/nrbquno4P1qtlpZXabspo9CCn1spJtbH+75HfvYDLf2zvTy9MNTNoffn9Pitt2Xmv6vjvLod1fNfWvnahM+blsKYUq93XmC7o73qZK02uu/ceK1+TT+fouDjNIsgjYSIiooCwESYiIgoIG2EiIqKAlJwx4QRzyUNi04Y6Xvd0spVtyonDdVwzMbpLjxanp1vLt71rbjjfdLAZ2/IfTSy5Ks7ZqOMvDlS00nqUM3fBeb/tOCvtkofv03H94X/qWB0+jJgduy/nPAAuT9luL/d90Scn0PrV/r5pdDTvncnK/7DYSsvauzc0e1iZO3fqePlZ5a20gT+doeORtX/R8d2T3rfyjTrVjCtnbrPrm/LmQE/7UrK3nzTjwLcu/5eOUz60x/iL0ziwF4+EiYiIAsJGmIiIKCDFqzva2+XcuL6VtHSouXn8ks6vRygkfBf0viy7m3N+uplB6/Hrb7bSGk7jJSjRyli9VsdjzzzdSrv7KVOHf5831kqb7+n6ndnHdFStTq9m5fu/767Q8cQLXtHx29vt15o6xdxAfMopz4SsZTnkVtr7/azlZsPMzQWKa7darNYOMXXS+cLZOl75VewDOVn77RuFrOpkuqdf+D1Nx3dVXmbl+/BLs99vvLCqlcbu6bwRZe8BWzPNd+6ght/p+IUOV1v5Sq/crOOMfzbk09oVPB4JExERBYSNMBERUUDYCBMREQVElCq4EapzEi7P1xdbOfw0HS/q7X+5SLTe3WumL3z3ynOstKy/FodmL9S+z/pQ4l1mftend4z/n/vtyxqqdDaXNn3W8m0dV0wog4ISeknVkBev03GtUSHnBcR5P8uP+gQKoE495OTjrOUzx83U8cQvOuu44UP5e47Fti/MmPCMdu9aaQkwb/Nxr9iXmdV7Ir538imS+2g8tDefg6vf+lbHbZPXWdnmHa6j46Hf97LSmnx0RMcJP3umGC7A9i1UtPXJI2EiIqKAsBEmIiIKSJG7RMnbhbX0dnu2q/nnjvAs+d/lyKvF/26yltOGmFmasHmbDrP2FGz384FLTffrjuamW7busOJzM+scZZn75dT5T8h2/8eEvbqam76vP7u0le3Zyyfk+mW7ldttLSfBc+mbmN+tj464zspX6+USVDdxsKZ7BWv5q6qLdPzRprMLbD0OzDSXtSW0s3sQvfWNfBkAIPwxX4fvHlvbxKhtZdt3uflOLHe1PavdN+9M1HGPFp10nLlnDwo7HgkTEREFhI0wERFRQIpEd/Ty8Sfq+J2OZuakE5NDc0bXBe1Vpox984UtZ3lv6G7f3D1aOzsd0vFjJ3/um2/yFjNL0750e2PGN3lOx+/vNtv/v2HR3VSiJCn1X3MDh0b/tdNe+r805FbVlbOt5VM9VbM766COy+zkLTliISEnrmZ55hJrfbXpmt42xh5iUOlHEE9HKpt6zAqdz0x56rjwn2dcrHlv6JDycaKV1vTlvjpOHGKOLZvcMyP/VyxGPBImIiIKCBthIiKigLARJiIiCkjRGBM+5zUdx3sUbs4pb9kPnBI+X364IuVb37QhW8wdZf7q1diTsjr/VohytOCIGSCu8pt9J5eMgl6ZYmxCg6k6bjbOvktZ81vNeHHWoUPwk1gpVccHT7XPDVjbzYwpjuox3reMIVvb6LjRRHsGJ9Z3gDyXLwJAlTq7dNyksrm7lX2xYeHEI2EiIqKAsBEmIiIKSJHojp5h7quN9kddllS4LTxiOq1+OtDcSvtqk5n9a++bday0Kn9s1XHmyhX5tHYUzjXf9bWW/+4xRsfey5U2nF/Xylfj5bX5ul7FTaNJ/1jLfbp30bG3O3ppl9esfCe8Y2YqS19kbqLRvftMK9+xZf/W8U2pU+EnXZmuzXkhVz/9dk97HSet+RNUOBzpdpK1PKXtSB1fefNAHZfGdhR2PBImIiIKCBthIiKigLARJiIiCkiRGBN+9Jo+Ov77NnNpwcQOr1v5KiSYAZ0WpewpLLM8FzfNOmxPeRarQ8p+rUEv3qbjykvMtJjJU+wxqwSYSx5SYV/+YJ+ATwWp9lT7t2lWDzNfofcm77ta2xep1Mjf1Sp2MlatsZZ3XNdIx9dO8IwPN7TnIrUuK4xwSaG3riJd2njyS3fpOPQuZUngOHBhkVjRjP93eWa6lXbKV3frOO37olVnPBImIiIKCBthIiKigBSJ7mj59S8dN/P0Fj2Gdla+xKamO2vFDbWstIQjpmuq/mP5e/P1WuDN3YuylA/sO6/0u6ejjsfUnabjxFT7DlwUm8y/V+l4+xnm8QtxYpjc8VOX+2uhlFi1irW8+MmmOn4p9Wsr7X+TTjULWUVrMI9HwkRERAFhI0xERBSQItEdHS1vd1bDwasi5CSK3tpT9uv4fM8QSBPMCWJ1iIqEvVeZLuK9dc3xnorQ6hxsfVDHT578qZXWK+V7Hbd/8n4rrcb0ojukwCNhIiKigLARJiIiCggbYSIiooAUqzFhIiIqHCr9uUXHR8rX1PH+8/ZZ+d48cbyOayeZMeGP9hxv5Ws/7God13i56I4Bh+KRMBERUUDYCBMREQWE3dFERBR3mctX6riqNx5n5xsS5YxoNYrpzGY8EiYiIgoIG2EiIqKAsBEmIiIKCBthIiKigLARJiIiCggbYSIiooCIUirodSAiIiqReCRMREQUEDbCREREAWEjTEREFBA2wkRERAEpco2wiAwVkXQR2Sci5aN8zgoROSIikyLkUSKyX0SejN/aFjwRmSoih0RketDrEi3WaWRFrU5Zn5EVtfoEWKc5iaVOA2mERaSFu9K7ReRvEemZyyLeV0qlKKX2u+VVEpEJIrLF/RvqzayUagLgqSjKbaOUesiznmNFZKmIZInIDWG2424R2eRuxxsikuxJaygiP4rIARFZIiJd/V5URJLd5+9xyxvkSasnIjNEZIeIPBfyvG9E5KSQbT0bQN8otjWu3J3T+5cpIqNzUURonU4JKe+IiMzPzsw6zX8icpWILHa/JFeISIdcPN2qT7e8diLys1ufm0VkYHYa67NgsE6tcgpFnRZ4IywiSQA+A/AlgCoAbgUwSUTSYih2BIByABoCaA/gWhHpE+OqAsBfAPoBmB2aICLdADwAoIv7uo0BPOrJ8i6AOQCqAngIwEciUt3ndYYCaAagAYDOAO4XkfPctAcBTADQCMAl2ZUvIlcCWKmUmpX3zYsfd+dMUUqlAKgJ4CCAD2Mor3tImb/GUp4H6zQKInIOgKcB9AFQAUBHACsjPilyedUAfAPgVTjvX1MA38W+pqzPaLFOjzIUhaFOlVIF+gegNYB9cK9Rdh/7DsDjUT5/KIBJIY9tA3CyZ/n/AEzL6Xkh6QpAU5+06QBuCHnsHQBPeZa7ANjkxmkADgOo4EmfBqCvT/n/ADjXs/w4gPfceAqA5m78HoArAFSE80Gr5FPeDQCmF3Tdel7/ejg7t0SZP6e6aQggE0Aj1mmB1eGvAG7K43PD7aNPAZgY4+eA9ck6LXZ1GkR3tPg89v/t3Xl4lcW9wPHfZCEhrGnKXnZI2MUVFBGtCgJFRQULKNqidUHBW8Tt0kvLRa0iWqFQvRepLNoKis8tXrXiLiiLuLCoBBCoCASBsCSELXn7xznMvHOSE0+Sc85k+X6eh4ffZObMeZNJMnln3pnpphNKHVRKXViBeq36YqSrBP5iO+1LEWmilMoI5n3red6RkPyuoZUopdJFpHkJdZ0uu0FELldKNRSRc0TkKwl8s/zJ87yDUfpcou0mEZnvBb8zRcrdpqeNlsAfVduicnXh0aYiopRKlMB1NVKB6aKdSqk/K6Vq+8qUtT17i8gBpdTHKjBltFQp1Sra1x6C9gyiTW2VqU1ddMLfiMheEZmolEpWSvUXkX4SGE4WERHP8xp6nleWCe43ReQBpVQ9pVQHEfm1v74YqSsih3zp03G9EvJO59cLU4//9aFlHxWRviLygYjMEpFkEekhIkuVUi8G52PuKu8nEW3BH8J+EhjK0crRpn6jReT5Cl5aJGjTgCbBa7pOAtfZU0TOFJFJpwuUoz1/JoE/zsaLSCsR2SaBocNYoj0N2rR4Pf7Xh5aNW5vGvRP2PO+kiFwtIoNFZI+ITBCRRSKyswLVjpPAHORmCcw3/620+pT90M+ocr5nngSGJ047HR8pIe90/hEpLi/k9VZZz/MOeJ53ved5Z4jI0yIyU0TulsC8yAYRuUxEbldKdSnn5xFtoyUwJBOVu9bgX+ZNReTlHylHm0ZPQfD/mZ7n7fY8b5+IPCkigypY56ue563xPO+YBObxLlBKNSipMO0ZdbRp8Xr8r7fKxrNNnTwd7XneOs/z+nmel+F53gAJTK6vrkB9BzzPG+V5XlPP87pK4PMKW59nP/TzQjnfdqOInOFLnyEiOZ7n7Q/mtVNK1QvJ31jCteSKyO4S6ipWVgIPsa30PG+DiHQXkU89zzshIusl9sPvkRotIXfBFXSTiCzxPC+vtEK0afQEr3+nBObromVdSH2n45Kmp2jPKKNNi11LpWlTV0uUeiilUpVSaUqpe0WkmVRguFEp1V4plaGUSlRKDZTAF21qFK6zllIqVQLfVMnBaz79NZsvImOUUl2C8wuTJPg5eJ6XLSJfiMjk4GuGSmAo45UwbzVfRCYppdKVUp1E5FYJ+XoopRqLyFgJPOggEhj6uUQpVVcCcxblfsoxWpRSF4hIC4nOU8wSnK8aJlEciqZNI/ZXEblbKdU4+LW4RwIrGipS31ClVE+lVLKI/E4CIyYVml+jPcuENrVVjjYt65Nc0fgnItNEJFcCQwJvSMjTccGP9w3z2t9L8af0hovILhE5KoFGGBDJ60Lyiz2lJyLvBz/u/3exL/+3IpIjIocl8A2Z4strE3x9gYhsEpHLfHmjRGSjL50iInOD9eSIyG9LuL75IjLMl24pIquCX8fpIWVvFgdPR0tgqUKJT0uWtU2DHx8hIjskzFPWtGlM2zJZRGaLyEEJTBvNEJHUCrbnHRJ4IjVXRJaKSEvaM64/n7RpJWzTuH0DRPEbaZKI5Ae/kepE+JpNwW+wuaWUOSaBifmIlkpV1n8iskwC8xrvuL4W2rRmtintWb3akzaNbZtynjAAAI5Uub2jAQCoLuiEAQBwJCmeb3Z5wjDGvh1ZVrS4xGUDFUF7uhOL9hShTV3iZ7R6ibQ9uRMGAMAROmEAAByhEwYAwBE6YQAAHKETBgDAETphAAAciesSJQCIpq3Te+u4ZY/dVl6ty3fE+3KAMuNOGAAAR+iEAQBwhOFoAFXK8YHn6vjdYU/o+Ivjja1ysyQzbteE2Etq0VzHh89raeXtuvaEjkd3X6XjyY2+ssp1mXWnjls+/HG0L7FcuBMGAMAROmEAAByhEwYAwBHmhAFUKUdaml9bTRJTdLww5/yQkvvjdEWIltybTRseHpRn5T3U4w0dj6i3NKL6ToacITV3zEwdT3747HJcYfRxJwwAgCN0wgAAOMJwNIBqYd07WVa6tVSOJSiwqXO6WemL5q7R8YSMGTpOiME94gNbrtVximyPev3lwZ0wAACO0AkDAOAIw9GlODHgHB1vH2r/vfLoJYt13CZ5n5V3z3+N1XGDhStjdHUA/EOWyfkOLwSl+uEO89Tzmw89YeWlJ6T6UuHvC0dt66/j9cvsqYeMjYU63tPL1DF76Byr3OElzXTciOFoAABqNjphAAAcoRMGAMCRGjknnH9tLx0fufGwlbfsbDOHUFet1nFaQi2rXKFX5EspKy/nQpPXYGFFrhRaQqKdLiosudx53a3krovq6biwt2nr4zvrWuUaZ/2g48NHU628thkHdLy/IE3He3alW+WS9iXruN52+7KafHJIx3t7NTAfX37AKle4cZOgdMcGmHYsEvOz1vxxliRVJpuf7q3j1deYeeAGCaklFRcRkQ+Pmd+zU8aPsfJq//MLHbc6Gb6tCwebnbD+8+FbrLxGf/2klCt2gzthAAAcoRMGAMCRmjMc3buHDh+b9hfz4ZTQgrV19HRuBx0nqCKr1N0Nvw37Vgv7P6PjR1peqeNT3+2M9GohYrVZj9nrraz3Z5olD/ktzHTAi2Oessp1r5UsMdPjx4v8mD5fDrfSDQZVvM7qZv8Y+2CGV8/xL3Gxp4ngjn8Zkkj4Ieij3gmrXK/nJui43YxvdJy6f7VVLuQshrBS6pj6G7+718o7FWEd8cSdMAAAjtAJAwDgCJ0wAACOVNs54cT69a30ZXOW67j4PLDReb7ZcrL91HVhy337QSMdP9VslZXnr78gq4mOk5kTLk6Z+dxDI3tZWdP/e7aOi7XZ1LVhKozhHHCICXvOs9LTm64OUxIVkXCNvS1s6yTmgSsL/4lIodtRhluK1PP1cVY6c7JZbhRm4WGZtB5unh+pjHPAobgTBgDAETphAAAcqbbD0bsXNLfS49Pf13Ged1zHPV8bb5Xr/OhGHRfmhz+W5eNnfI/j/2FV2HLf9TdDZ+3eDlusxjpyvRmCXvH47FJKRmZZQW0rPWXSr3Scm2X+5jyVddQqV7THDJ0VpdnL0WrtM7t1FbUr0HGbJvvtN29a9uvduzXDSjeQLWWvpBpKbNJYx7e0W2Hl+U9O6vPQXTpOl8q3G1J1d9HcNTpOL2UnrEf2mZ3sOt+XbeVFYwi6KuNOGAAAR+iEAQBwpNoOR1/b9ouweed+dLuOM2+3n2iNdGjkp2vNJvL7iwqsvIwEMyT684vNdWyPsO6a5PIHP6pwHZduvEbH6slGVl69N1eauMLvJJKYbg5t2DGneSklw+v8oRki7ziOJ6pLsuPXZre6m+r/v5U3fldfHWcs2aBjexIB8TC6oX+Vgj0V5D+MYdXwLjouPMiUix93wgAAOEInDACAI3TCAAA4Um3nhN9+oK+Vvvt/PtfxlLP+oeP5mT+3yhVmb42ofu9zs5RpT6F94HyG70+bL/e10DHLT4r7QyPfkrBSjkl5bH9nKz3ng4t13GnqNlNHzhqJtoRU3/KlV9J0vC5rQcR1ZP39Th23n+i7Ri/Ss2Fqlhm3PBs2b01OKx3/5Eh22HKIPv8OWSIiddTHYUqK3LbkNzpuvyl+y8cKLzlLx0WJKmy5WgePW2nv0w1hSsYWd8IAADhCJwwAgCPVdjg65XV7WNK/M9aWIc/o+JEhTaxyzaZHNhx97Bdm8/52SSvDlsv93CyZYTi6uKn7Ouk4M3W3lTd54Sgdt33W/tp1zDG7lEV7x53ErllW+sA08w4rshZFVMewrQOsdMffmcNAiopq+h5BP+6iVHMwe+jSo1rz0iWadkwxu9/1vHSTlTeycfif7VkdM6N6HVXBwU51rXRaQvwOTPE7PvBcK504IUfHSzuZ3+/Jyp4q9Nt26piVHrTc7L7WftTnocVjhjthAAAcoRMGAMCRajscHarzfd/oeHbftjqeevvzVrkJjW7ScYe/H9Lx95c2tMr1Gv6ljmsr+3zTqzYP1nH7x3xP/5bxmmuCTwa00fFHu+0N4FtJdM8ZLU1CNzMs3vDZHCtvaZt3Iqoj6/0xJn7gByuv6OgPocURwj8snKzMTnOL8n5ilWu41rRPaefFJrUwO5ptubO1lff6DdN03CbpMx2vPWF/p7VLMsPioefjzirlvaurggz7vi0hjvdx/iHo1/53ppWXovzD4uGHoP3aJtnt+VFfU+eYJmYXvsKcvWW4yrLjThgAAEfohAEAcIROGAAAR2rMnHDhYXPq0dLfXKLjKQues8p9c6NvpufG8r3Xlg/b6Lj1YQ4aL82p3XucvK9/DlhEJN03D7yglDngvYVHddznvXFWXuYtZsedUydPCMrIM7sbnfTM3OxTWy+zitX/tuRlhF6fnlb6tufNcrIBaYdCSqfoaPwuMxf95aN2HWc8aOampzdfHubCa44G2+1Z+OPeSR3b87IVp87tbqVHPmlO0yrtvXYXmlPtrvr8FisvLcX8XL7ffbGV99NEcwrUruvNKV5NZjAnDABAtUQnDACAIzVmONpPrTBDTJPbnW3l5V/XS8ffDzZDLw3XpFjlDnU1w2Wbr/5LtC8RMZCQZg5fyJ9ub97+WoTLkPqtGKvjjjd9ZuVxFENsnFzSOOQjJQ9H3z/fPlDjwlSzI9IbR+1dtqY9ZHZjq7vY7L5WMMa+LxnX+F0dzzlkT2HURLX/b7WVPjjT/I5skmgPEbc/67sKvdfm8Xb39Kv64evzD0GPmDBBx419bSsiktDD14Zv2HUU+fZmS8qP308zd8IAADhCJwwAgCN0wgAAOFIj54RLU+dlM4eQ+XL4crV/2dskro7hBaFCklq31PH2kSZe1+3PEdcxdMsgHXeYnKdjtiGtXPwnL4nYpy/dt9heb9h2sVk6mJhhtsUcMu4Dq1zrJLMl7ZxZQ6y8xhL+QPua4uUj3XQ8tqE9V78g8yUdX3/FPTqu9aZ9wp1fQs8uOp7R+29hy/3rVIGVvuGBe3Vcf7E5+Sqprb1d6fkLv5BwcgrNcyIZz8VvaSl3wgAAOEInDACAIwxHo1r7eqI5SWfz0MiGoJcV1LbSp26to+PC7JKXx6DiWnxghgOTbzUn4Txy/xyr3BMbR+pYfWxOM0sQJTZzj9G6104rZ8tTZjope/hsHReFLDTrttycjNVmFsPPoebNNlM1Yx+yTzZK9586NcHsOqXetrsd75RZ5rT9SnNaXf/a+WHfd2leNyvd8B/rdXxgtNkBbci971nl7s/YKOEMnnGfjpvHcaqBO2EAAByhEwYAwBGGo1HlJaSaYa/skA34V1813Zeyh5n9ns41G7a/dfMFVp6XvSG0OGIg6d21On7rqNl9qV/to1a5o/Ne0fEj2QN1XCRrrXL+HZBe67TEfjPfxkm5Reap6vNfuNcqljlzh47towsgItLsxa91PO22LlbexIyvdPxWF/P17zRtrFWuw3+Yp5lb9ftXRO87qK49rPzdh+YJ9z82jWzaacimK610y+fM5xLPlQ/cCQMA4AidMAAAjtAJAwDgCHPCqPK2Tj5Tx9nDZ4XkljwPvLvQnmd8ZUp/Hdf9dGVoccTZE6PNMqQ5j9vLixa0fVPHA3u+6Msp3z3FJTMm6rjdNHtpCvPApSvMzdXxS3MvtfImTvwqtLiIiKwfNsNKLxr4Mx1fVcf/9bdPrvNrm5Rqpf/YtORduI57J630yK1me8OEa+3fAf7PJZ64EwYAwBE6YQAAHGE4Osb8m8MX7j/g8Eqql7xhvXS86gb/MqTU4oWD/EPQ1/xuopWXvih+G7bjx/l3wjrSP83Ku+yKu3S8c4BZhtSn+2ar3Ir1HXVcZ5t94HyrJTk6bp7NTljR0PRP9tcxs8dtOs4e8KyOk1WiVW5Uvd2+VPgh6EhN3GN+N7w3/zz7Gp+ufG3NnTAAAI7QCQMA4AidMAAAjjAnHAOPj5in4yPXmyUyM6cMs8o1eIGlMJE6bzj/TAAAAlhJREFU9gt7bufOqS/ruH5C+HngvWHmgdPnMQdcVRQdtZeSpC1ZpeNM326UP4S8LlPCHx4fz20Ja6qsO8zJRl1/b+bx3xg5zSrXKin8drJ+zxxsp+On3r3Cymv/kjmBK3Gl2Wa26anKNwccijthAAAcoRMGAMARhqNjYHBani9l4kl9i6xyDV6I0wVVUYlds3TcYZK9+84v64YOPgbsDdkJ6+pJviHo+QxBA/HiHTdDxG0fND97dz54YYXr7iirwuZ5Fa49vrgTBgDAETphAAAcYTi6nNKXf6fjb04et/I6JZe868uZXbZZ6fzoX1aVppLsb8cD08wzrEtbfhj2dfuLCnTc51X7UPaODEEDqMS4EwYAwBE6YQAAHKETBgDAEeaEy+nUzu91PGLmBCvvn+Mf1/GkXWZnl/x7m4bUUvIym5pK1bZ3zllxxqKIXjf861E67jieXcgAVB3cCQMA4AidMAAAjjAcHQXNptubhN883b8jjH/3rPWC8Iry8qx0p4VjdTz9mnlW3sTPrtNxmxEbY3thABAj3AkDAOAInTAAAI7QCQMA4Ahzwqg8PPv8k3b3my0nZ92faeW1kXVxuSQAiCXuhAEAcIROGAAAR5TnVbUjkAEAqB64EwYAwBE6YQAAHKETBgDAETphAAAcoRMGAMAROmEAAByhEwYAwBE6YQAAHKETBgDAETphAAAcoRMGAMAROmEAAByhEwYAwBE6YQAAHKETBgDAETphAAAcoRMGAMAROmEAAByhEwYAwBE6YQAAHKETBgDAETphAAAcoRMGAMCRfwOduaSphdFr2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import random\n",
    "with tf.Session() as sess:\n",
    "    ckpt = tf.train.get_checkpoint_state('./model/')\n",
    "    if ckpt and ckpt.model_checkpoint_path:\n",
    "        saver.restore(sess, ckpt.model_checkpoint_path)\n",
    "        final_pred, acc = sess.run(\n",
    "            [pred, accuracy],\n",
    "            feed_dict={\n",
    "                x: mnist.test.images[:-1],\n",
    "                y: mnist.test.labels[:-1],\n",
    "                keep_prob: 1.0\n",
    "            })\n",
    "        orders = np.argsort(final_pred)\n",
    "        plt.figure(figsize=(8, 8))\n",
    "        print(acc)\n",
    "        r=random.randint(0,9984)\n",
    "        print(r)\n",
    "        for idx in range(16):\n",
    "            order = orders[idx+r, :][-1]\n",
    "            prob = final_pred[idx+r, :][order]\n",
    "            plt.subplot(4, 4, idx + 1)\n",
    "            plt.axis('off')\n",
    "            plt.title('{}: [{}]-[{:.1f}%]'.format(\n",
    "                np.argmax(mnist.test.labels[idx+r]), order, prob * 100))\n",
    "            plt.imshow(mnist.test.images[idx+r].reshape((28, 28)))\n",
    "\n",
    "    else:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>卷积的特性描述"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1、局部相关性：相邻像素结合起来表达一个特征，而距离较远的像素相关性较弱；\n",
    "          随着卷积层的加深，feature map上的一个点映射到原图中的感受野越大\n",
    "         卷积不仅仅适用于图像，具有局部相关性的数据都可以使用卷积来处理\n",
    "         带来一定的遮挡不变性\n",
    "2、参数共享，卷积核参数共享，权值共享\n",
    "3、平移宽容：在任何位置都可以激活神经元\n",
    "4、缩放宽容：一定程度上图像缩放（一般是25%以内）不会产生太大的干扰\n",
    "5、少许降维：一幅图像经过卷积后，特征图的尺寸会减小，减小原图中冗余的信息特征。\n",
    "6、对输入的尺寸不做要求：\n",
    "  在全连接中，权值w的尺寸是有输入决定的\n",
    "  卷积核的尺寸只与网络设计有关，与输入没有任何关系。\n",
    "  卷积核固定的情况下，feature的尺寸是有输入决定的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>卷积的效果为什么比全连接网络好"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1、首先卷积的一个重要的特性是局部相关性，相邻像素结合起来表达一个特征，而在全连接网络中，每一个神经元与上一层所有神经元连接，每一个像素结合了图像中所有像素的信息，而在一幅图像中是存在很多信息，距离较远的像素信息可能会对最后的特征形成干扰，影响识别效果。\n",
    "2、网络层数的限制，一般来说，网络层数越多其表达能力越强，但是对于全连接网络来说，通过梯度下降方法去训练深度全连接网络会产生大量的参数，网络训练非常困难，需要耗费大量的资源和时间，所以全连接网络的梯度训练很难超过三层，因此，我们不会训练一个网络层数很深的全连接网络，这就限制了它的能力。而卷积神经网络的两个特性是局部感受野和参数共享，这两个特性大大减小了训练的参数数量，可以训练层数较深的网络来提高模型性能。\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
